Apparently I’m not the only person to reach this conclusion, either.

I spent two years reading all the self-help books I could find. As of a year ago, I had read 340 self-help books. Because I’m insane.

My conclusion from all that reading?

95% of self-help books are complete bullshit.

To be clear, I am all for self-improvement. Reading books, blogs, and newsletters by people who have accomplished great things is a fine way to research your own path in life. But these people, however famous and important they may be, are not going to help you.

Unfortunately that’s not the answer he wanted. To him, my answer [that nobody was going to help him become successful] was really discouraging. To me, if I was receiving that answer from someone else, it would be really encouraging.

I like being reminded that nobody’s going to help me – that it’s all up to me. It puts my focus back on the things I can control – not waiting for outside circumstances.

Take it from The Ultimate Productivity Blog:

Reading self-help advice from other people, however well-intentioned, is no substitute for getting your own damn work done. The sooner you come to terms with this, the better off you’ll be.

Get out there and do stuff because you fundamentally enjoy it and because it makes you better. As a writer, as an analyst, as a techie, whatever. Learn to love practicing the fundamentals and do it better each time. Over time, quality does lead to success, but you have to be patient. Really patient. Turns out, “overnight” success takes years. Maybe even decades. This is not a sprint, it’s a marathon. Plan accordingly.

For example, I don’t care if anyone reads what I write here. I’m writing to satisfy myself first and foremost. If others read it and benefit from it, fantastic — that’s a welcome side effect. If I worry about who is reading, why they’re reading, or if anyone is even reading at all, I’d be too paralyzed to write! That’d be the least productive outcome of all.

That’s not to say that some introspection about the nature of your work isn’t useful. It is. Even the weary self-help student I quoted above concluded that 5% of self-help advice surprisingly wasn’t bullshit. The one book he recommended without hesitation? 59 Seconds: Think a Little, Change a Lot.

Despite my deep reservations about the genre, I ordered this book based on his recommendation and a number of credible references to it I noticed on the Skeptic Stack Exchange.

Why does this self-help book work when so many others fail? In a word, science! The author goes out of his way to find actual published scientific research documenting specific ways we can make small changes in our behavior to produce better outcomes for ourselves and those around us. It’s powerful stuff, and the book is full of great, research backed insights like this one:

A group of participants were asked to select a negative experience. One group of participants were then asked to have a long chat with a supportive experimenter about the event, while a second group were invited to chat about a far more mundane topic – a typical day.

Participants who had spent time talking about their traumatic event thought the chat had been helpful. However, the various questionnaires told a very different story. In reality the chat had no significant impact at all. They might just as well have been chatting about a typical day.

In several studies, participants who have experienced a traumatic event have been encouraged to spend just a few minutes each day writing in a diary-type account of their deepest thoughts and feelings about it. For example, in one study participants who had just been made redundant were asked to reflect upon their deepest thoughts and feelings about their job loss, including how it had affected both their personal and professional lives. Although these types of exercises were both speedy and simple, the results revealed a remarkable boost in their psychological and physical well-being, including a reduction in health problems and an increase in self-esteem and happiness.

The results left psychologists with something of a mystery. Why would talking about a traumatic experience have almost no effect but writing about it yield such significant benefits? From a psychological perspective, talking and writing are very different. Talking can often be somewhat unstructured, disorganized, even chaotic. In contrast, writing encourages the creation of a story line and structure that help people make sense of what has happened and work towards a solution. In short, talking can add to a sense of confusion, but writing provides a more systematic, solution-based approach.

Therefore, the real world change you would make based on this advice – the proverbial 59 seconds on the book jacket – is to avoid talking through traumatic experiences in favor of writing about them. Not because some self-help guru said so, but because the published research data tells us that talking doesn’t work and writing does. Not exactly intuitive, since talking through our problems with a friend always feels like the right thing to do, but I have certainly documented many times over the value of writing through a problem.

59 Seconds is so good, in fact, it has rekindled my hopes that our new Stack Exchange Productivity Q&A can work. I’d love for our productivity site to be founded on a scientific basis, and not the blind cult of personality I’ve come to expect from the self-help industry.

Remember, nobody’s going to help you … except science, and if you’re willing to put in the required elbow grease each and every day – yourself.

Go to Source

• Building a PC, Part I: Minimal boot
• Building a PC, Part II: Burn in
• Building a PC, Part III: Overclocking
• Building a PC, Part IV: Now It's Your Turn
• Building a PC, Part V: Upgrading
• Building a PC, Part VI: Rebuilding

While the advice in those original articles is still quite sound, my old 2007 era case was feeling mighty creaky. I needed a new chassis. I also wanted a motherboard that supported native 6 Gbps SATA for the latest generation of SSDs that truly benefit from them. The buzz around the Sandy Bridge based Core i7-2600k was nearly deafening, and I’ve fallen completely in love with my last HTPC build based on the same technology. (Oh, and even if you already read that article, read it again because I added new PicoPSU and case information that takes it from awesome to sublime – on the order of 17 watts idle!)

So I decided it was time to build myself a nice Sandy Bridge system. What I ended up with is easily the best case and motherboard combination I’ve ever laid hands on. Read on!

I cut out a lot of the initial research work by relying on my old, dear friends at Tech Report and their current workstation recommendations:

• Asus P8Z68-V Pro $220
• Corsair Vengeance 16 GB kit $155
• Core i7-2600K $315

As for the case, I was impressed by the Tech Report review of the Corsair 600T, which even comes in a heart-stopping pseudo stormtrooper white. WANT.

When it comes to power supplies, I’m crazy about efficiency, and fortunately there are now lots of so-called “80 Plus Gold” PSUs out there now, offering a staggering 90% efficiency under most loads. Power supply efficiency is important, because the rest of that heat is dumped back into your case. The less efficient your PSU, the more heat buildup you’ll have under load. I chose the Seasonic X-760 – which, when bench tested, indeed delivered the promised 90% efficiency – but any high quality 80 Plus Gold model will generally do.

The CPU (and possibly, depending on your tastes, the video card) is the biggest heat generator inside your PC. The better and more efficient the CPU cooler, the quieter your whole system can be. This also affects how much you can overclock. I chose the Thermalright Venomous-X Silent Edition on the basis of it being the current top dog for efficiency, and because it had a simple mounting system. Large coolers can be a real bear to install. And did I mention it comes with an especially quiet fan out of the box, too?

Once I had all the parts in hand, it was a simple matter of building it up, as documented in my previous post series. I adore this Corsair case; it is an absolute joy to work in. Everything in it is cleverly designed, from the rear cable routing area with rubber grommets all over the place for easily passing cables back and forth, to the tool-less 2.5″ and 3.5″ bays, to the super easily removable side panels. It’s like they read a giant list of all my prior complaints with every PC case I’ve ever used and fixed every. single. last. one of them.

The end result is what you see here:

There are some significant tweaks visible in the above picture that I do recommend:

1. Use tin snips to remove the rear exhaust grill. We don’t need it back there, and the exhaust airflow is critical. Fan grills affect low-speed fan airflow more than you’d think:
   

   Wire grills also have an effect: ~20%. This was checked with an anemometer on several different fans of 80, 92 and 120mm size, at full and lower speeds. The airflow reduction went as high as 24% but it was never below 19%. At 12V, the reduction in airflow with most fans will be relatively harmless, though there is an increase in turbulence noise (audible to me). But at the low airflow rates SPCR members run fans, I think the airflow reduction is significant.

2. Install a 140mm rear exhaust fan. The Noctua NF-P14 is expensive at $25 but is cleverly designed to give you 140mm of super-quiet fan in the space a 120mm fan would normally take. It just barely fits in the rear exhaust fan slot with a little nudging. But it does fit; it’s the beige fan in the above picture. It also comes with its own speed reducers and accessories.
3. Use fan speed reducers on all the fans. The case has two 200mm fans, and the 140mm fan we’re adding. I couldn’t get the Asus motherboard’s “QFan” fan control system to work, as it seems to require 4-pin fans, and all the ones I had (including the ones that came with the case) are 3-pin. While I do prefer dynamic, temperature based control when I can get it, the next best thing is to use hardware to slow down the fans. I like the Zalman-ZM-RC56 resistor connector as the simplest solution, but it’s getting hard to find for some reason. The Zalman Fan Mate 2 will also work, and allows you to individually adjust the speed of each fan. The case also has a built in fan controller – that’s the knob you see on the front top – but I found it too limited in range for my purposes.
4. Add acoustic foam to taste. Between inexpensive eggcrate foam and thin, adhesive backed open-cell foam, you can easily reduce that last 10-20% of fan noise to a very pleasant white noise. It works well in the areas pictured, and also on the interior of the side panel “facing” the fans. See item 6 in my Building a Quiet PC post for details.

And then, of course, the overclockening. What kind of geek would I be if I didn’t attempt to turn this baby up to 11? This is another area where Sandy Bridge is a revelation: so long as you buy one of the blessed “K” series processors, overclocking is as simple as setting the multiplier to the desired value. It is ridiculously simple. And my results, for once, were immediately as good as the ones everyone else was crowing about: 4.4 GHz totally stable!

(beware: there is one nasty little issue with the Asus motherboard’s auto-overclock feature. The PLL Overvoltage setting, which auto-overclock “helpfully” enables, completely bollixes up resuming from sleep. Just turn it off, and all is well. I don’t even want to tell you how long it took me to figure that one out.)

The total package with a nice SSD delivers a near-perfect Windows Experience score:

I won’t lie to you. This is not a compact build. It’s big! Those roomy side areas come at a cost, and that makes it a very wide case. But that’s to be expected for a desktop powerhouse machine. And since my last case lasted me from 2007-2011, I’ll happily accept a little bulk for something that’s easy to work on and upgrade over time.

It’s a fantastic new reboot of my system, and I didn’t expect to be this excited about the final result. This is not merely an incremental improvement over what I had, it’s much quieter, easier to work on, and when overclocked to 4.4 GHz, noticeably faster too. (I do slightly mourn the loss of 8 GB of RAM, but I’ll survive.)

In this build, I already had hard drives, DVD drive, a sound card, and so forth … but for completeness’ sake I’ll list everything here if you want to mirror this setup. Realize that some of this comes down to personal taste, so I’m just listing what I recommend. Feel free to change anything out, and bear in mind that Sandy Bridge has decent default onboard video as well.

• Asus P8Z68-V Pro $220
• Corsair Vengeance 16 GB RAM $155
• Core i7-2600K CPU $315
• Corsair 600T Special Edition case $170
• Corsair 850w 80 Plus Gold PSU $170
• Thermalright Venomous-X Silent Edition CPU heatsink $75
• Vertex 3 240 GB SSD $500
• WD Caviar Green 3 TB HDD $170
• Noctua Ultra Silent 140mm Fan NF-P14 FLX $25
• Radeon 6850 triple monitor capable video card $170

Remember, if you can put together a LEGO kit, you can build this totally sweet PC for yourself, too. Good luck and happy building!

Go to Source

[Google found that] the page with 10 results took 0.4 seconds to generate. The page with 30 results took 0.9 seconds. Half a second delay caused a 20% drop in traffic. Half a second delay killed user satisfaction.

In A/B tests, [Amazon] tried delaying the page in increments of 100 milliseconds and found that even very small delays would result in substantial and costly drops in revenue.

I believe the converse of this is also true. That is, the faster your website is, the more people will use it. This follows logically if you think like an information omnivore: the faster you can load the page, the faster you can tell whether that page contains what you want. Therefore, you should always favor fast websites. The opportunity cost for switching on the public internet is effectively nil, and whatever it is that you’re looking for, there are multiple websites that offer a similar experience. So how do you distinguish yourself? You start by being, above all else, fast.

Do you, too, feel the need – the need for speed? If so, I have three pieces of advice that I’d like to share with you.

1. Follow the Yahoo Guidelines. Religiously.

The golden reference standard for building a fast website remains Yahoo’s 13 Simple Rules for Speeding Up Your Web Site from 2007. There is one caveat, however:

There’s some good advice here, but there’s also a lot of advice that only makes sense if you run a website that gets millions of unique users per day. Do you run a website like that? If so, what are you doing reading this instead of flying your private jet to a Bermuda vacation with your trophy wife?

So … a funny thing happened to me since I wrote that four years ago. I now run a network of public, community driven Q&A web sites that do get millions of daily unique users. (I’m still waiting on the jet and trophy wife.) It does depend a little on the size of your site, but if you run a public website, you really should pore over Yahoo’s checklist and take every line of it to heart. Or use the tools that do this for you:

• Yahoo YSlow
• Google Page Speed
• Pingdom Tools

We’ve long since implemented most of the 13 items on Yahoo’s list, except for one. But it’s a big one: Using a Content Delivery Network.

The user’s proximity to your web server has an impact on response times. Deploying your content across multiple, geographically dispersed servers will make your pages load faster from the user’s perspective. But where should you start?

As a first step to implementing geographically dispersed content, don’t attempt to redesign your web application to work in a distributed architecture. Depending on the application, changing the architecture could include daunting tasks such as synchronizing session state and replicating database transactions across server locations. Attempts to reduce the distance between users and your content could be delayed by, or never pass, this application architecture step.

Remember that 80-90% of the end-user response time is spent downloading all the components in the page: images, stylesheets, scripts, Flash, etc. This is the Performance Golden Rule. Rather than starting with the difficult task of redesigning your application architecture, it’s better to first disperse your static content. This not only achieves a bigger reduction in response times, but it’s easier thanks to content delivery networks.

As a final optimization step, we just rolled out a CDN for all our static content. The results are promising; the baseline here is our datacenter in NYC, so the below should be read as “how much faster did our website get for users in this area of the world?”

In the interests of technical accuracy, static content isn’t the complete performance picture; you still have to talk to our servers in NYC to get the dynamic content which is the meat of the page. But 90% of our visitors are anonymous, only 36% of our traffic is from the USA, and Yahoo’s research shows that 40 to 60 percent of daily vistors come in with an empty browser cache. Optimizing this cold cache performance worldwide is a huge win.

Now, I would not recommend going directly for a CDN. I’d leave that until later, as there are a bunch of performance tweaks on Yahoo’s list which are free and trivial to implement. But using a CDN has gotten a heck of a lot less expensive and much simpler since 2007, with lots more competition in the space from companies like Amazon’s, NetDNA, and CacheFly. So when the time comes, and you’ve worked through the Yahoo list as religiously as I recommend, you’ll be ready.

2. Love (and Optimize for) Your Anonymous and Registered Users

Our Q&A sites are all about making the internet better. That’s why all the contributed content is licensed back to the community under Creative Commons and always visible regardless of whether you are logged in or not. I despise walled gardens. In fact, you don’t actually have to log in at all to participate in Q&A with us. Not even a little!

The primary source of our traffic is anonymous users arriving from search engines and elsewhere. It’s classic “write once, read – and hopefully edit – millions of times.” But we are also making the site richer and more dynamic for our avid community members, who definitely are logged in. We add features all the time, which means we’re serving up more JavaScript and HTML. There’s an unavoidable tension here between the download footprint for users who are on the site every day, and users who may visit once a month or once a year.

Both classes are important, but have fundamentally different needs. Anonymous users are voracious consumers optimizing for rapid browsing, while our avid community members are the source of all the great content that drives the network. These guys (and gals) need each other, and they both deserve special treatment. We design and optimize for two classes of users: anonymous, and logged in. Consider the following Google Chrome network panel trace on a random Super User question I picked:

We minimize the footprint of HTML, CSS and Javascript for anonymous users so they get their pages even faster. We load a stub of very basic functionality and dynamically “rez in” things like editing when the user focuses the answer input area. For logged in users, the footprint is necessarily larger, but we can also add features for our most avid community members at will without fear of harming the experience of the vast, silent majority of anonymous users.

3. Make Performance a Point of (Public) Pride

Now that we’ve exhausted the Yahoo performance guidance, and made sure we’re serving the absolute minimum necessary to our anonymous users – where else can we go for performance? Back to our code, of course.

When it comes to website performance, there is no getting around one fundamental law of the universe: you can never serve a webpage faster than it you can render it on the server. I know, duh. But I’m telling you, it’s very easy to fall into the trap of not noticing a few hundred milliseconds here and there over the course of a year or so of development, and then one day you turn around and your pages are taking almost a full freaking second to render on the server. It’s a heck of a liability to start 1 full second in the hole before you’ve even transmitted your first byte over the wire!

That’s why, as a developer, you need to put performance right in front of your face on every single page, all the time. That’s exactly what we did with our MVC Mini Profiler, which we are contributing back to the world as open source. The simple act of putting a render time in the upper right hand corner of every page we serve forced us to fix all our performance regressions and omissions.

(Note that you can click on the SQL linked above to see what’s actually being run and how long it took in each step. And you can use the share link to share the profiler data for this run with your fellow developers to shame them diagnose a particular problem. And it works for multiple AJAX requests. Have I mentioned that our open source MVC Mini Profiler is totally freaking awesome? If you’re on a .NET stack, you should really check it out. )

In fact, with the render time appearing on every page for everyone on the dev team, performance became a point of pride. We had so many places where we had just gotten a little sloppy or missed some tiny thing that slowed a page down inordinately. Most of the performance fixes were trivial, and even the ones that were not turned into fantastic opportunities to rearchitect and make things simpler and faster for all of our users.

Did it work? You bet your sweet ILAsm it worked:

That’s the Google crawler page download time; the experimental Google Site Performance page, which ostensibly reflects complete full-page browser load time, confirms the improvements:

While server page render time is only part of the performance story, it is the baseline from which you start. I cannot emphasize enough how much the simple act of putting the page render time on the page helped us, as a development team, build a dramatically faster site. Our site was always relatively fast, but even for a historically “fast” site like ours, we realized huge gains in performance from this one simple change.

I won’t lie to you. Performance isn’t easy. It’s been a long, hard road getting to where we are now – and we’ve thrown a lot of unicorn dollars toward really nice hardware to run everything on, though I wouldn’t call any of our hardware choices particularly extravagant. And I did follow my own advice, for the record.

I distinctly remember switching from AltaVista to Google back in 2000 in no small part because it was blazing fast. To me, performance is a feature, and I simply like using fast websites more than slow websites, so naturally I’m going to build a site that I would want to use. But I think there’s also a lesson to be learned here about the competitive landscape of the public internet, where there are two kinds of websites: the quick and the dead.

Which one will you be?

Go to Source

It turns out that having children is a great excuse reason to get into fun things like scooters. I didn’t know much about scooters for adults, so being an obsessive geek, of course I had to research the heck out of this topic. My research turned up the Xootr MG as a top choice.

News flash: scooters are fun. Really fun!

But per my research (and now, personal experience) scooters are also surprisingly practical forms of transportation in certain situations, namely when …

• you need to travel 1-3 miles
• the route is not too hilly
• it is not raining or wet
• the route is mostly paved or has sidewalks
• you are comfortable being “that awkward looking guy on a scooter”

Scooters are very primitive machines; it is both their greatest strength and their greatest weakness. It’s arguably the simplest personal wheeled vehicle there is. In these short distance scenarios, scooters tend to win over, say, bicycles because there’s less setup and teardown necessary – you don’t have to lock up a scooter, nor do you have to wear a helmet (though I highly recommend one). Just hop on and go! You get almost all the benefits of gravity and wheeled efficiency with a minimum of fuss and maintenance. And yes, it’s fun, too!

I’m just a scooter newbie, but the Xootr MG has a few characteristics I liked a lot, including rock-solid construction, a front brake (not super efficient, but reasonably effective when combined with the rear foot fender brake), and a wide, comfortable platform for your feet. But it does take some effort to kick around – don’t forget to alternate your legs – and the ride can be rough at times depending on the surface. Large bumps and very uneven surfaces are wreck material. And going uphill on a scooter, beyond the absolute wussiest and mildest of grades, is simply out of the question.

For longer distances, or if the terrain is rougher or hillier, a scooter might work, but it’d be a tough way to travel. What you need in those cases is a small, portable bicycle – one you can take with you. I’ve dabbled in foldable bicycles before, and we own two Dahon folding bicycles. They’re great, versatile and inexpensive bikes.

Dahon makes fine traditional folding bicycles, but they are not quite as pick-up-and-go as I would like for short trips. As an experiment, I purchased something I’ve had my eye on for a long time: the Strida LT folding bicycle. Or, as I like to call it, my “mid-life crisis vehicle”.

(also pictured: some cool accessories that I recommend for Strida owners: a Cateye Reflex rear LED light on the rack, Knog beetle silicone front LED light on the handlebars, and a Sunlite Bicycle bungie cargo net.)

The appeal of the Strida is that it folds down to an incredibly small size.

It’s almost a pogo stick in folded form. I took my Strida on a short trip into San Francisco for a speaking gig in the city, which involved riding on BART, and the Strida in practice is everything I dreamed a modern ultra-portable folding bicycle could be:

• front and rear disc brakes; superb stoppers
• belt drive so no grease on your hands or pants
• built in fenders in case you encounter puddles or rain
• comfortable, full size(ish) upright riding position
• super-easy, crazy fast folding: five seconds, no kidding!
• when folded, the bike can be propped by the rear rack (as pictured) or strolled along by rolling it on its wheels.

The Strida may look odd, and perhaps it is odd, but I found it to be shockingly close to an ideal go-anywhere do-anything convenience bicycle. It isn’t perfect, of course:

• My only real beef with the Strida: the seat adjustment is horrendously kludgey. Adjusting the seat height on a Strida is painfully awkward even in the garage; on the go it’s not an option.
• It is a small wheel bicycle, with all the unavoidable physical compromises that entails. It’ll always be a little twitchy and not something you would want to go on a 10 or 20 mile ride with.
• It’s a single speed, and you’re not supposed to stand out of the saddle for power pedaling at any time. The frame and belt drive won’t take it. On anything other than a moderate uphill you’ll need to hop off and walk. (There is a slightly fancier Strida that has two internal hub gears, but I know nothing about it.)
• Because the fold involves a ball joint, it is possible to permanently damage the bike if you aren’t careful when you fold and force it. I doubt this is a real concern for anyone who has folded a Strida more than once, but if a ham-fisted friend tries to fold your Strida to “test it out”, you might be in trouble.

None of these criticisms apply to the Dahon, so hopefully you can get a sense of the dividing line between an ultra-folder and a plain old folding bicycle.

Being a geek, it’s not like I spend a lot of time outdoors. But when I do venture outside, I like to travel in a manner befitting a geek. That is, with my utility belt fully equipped, and in the dorkiest, most efficient vehicle possible for a trip of that particular length. Scooters, folding bicycles, recumbents, pogo sticks … whatever it takes. If you, too, would like to geek out around town, consider adding the Xootr MG scooter and Strida LT folding bicycle to your stable of geek transportation systems.

Go to Source

Stack Overflow was always intended to be a democracy. With the Stack Exchange Q&A network, we’ve come a long way towards that goal:

• We create new communities through the open, democratic process defined at Area 51.
• Our communities are maintained and operated by the most avid citizens within that community. The more reputation you have, the more privileges you earn.
• We hold yearly moderator elections once each community is large enough to support them.

We strive mightily to build self organizing, self governing communities of people who are passionate about a topic, whether it be motor vehicles or homebrewing or musical instruments, or … whatever. Our general philosophy is power to the people.

But in the absence of some system of law, the tiny minority of users out to do harm – intentionally or not – eventually drive out all the civil community members, leaving behind a lawless, chaotic badland.

Our method of dealing with disruptive or destructive community members is simple: their accounts are placed in timed suspension. Initial suspension periods range from 1 to 7 days, and increase exponentially with each subsequent suspension. We prefer the term “timed suspension” to “ban” to emphasize that we do want users to come back to their accounts, if they can learn to refrain from engaging in those disruptive or problematic behaviors. It’s not so much a punishment as a time for the user to cool down and reflect on the nature of their participation in our community. (Well, at least in theory.)

Timed suspension works, but much like democracy itself, it is a highly imperfect, noisy system. The transparency provides ample evidence that moderators aren’t secretly whisking people away in the middle of the night. But it can also be a bit too … entertaining for some members of the community, leading to hours and hours of meta-discussion about who is suspended, why they are suspended, whether it was fair, what the evidence is, how we are censoring people, and on and on and on. While a certain amount of introspection is important and necessary, it can also become a substitute for getting stuff done. This might naturally lead one to wonder – what if we could suspend problematic users without anyone knowing they had been suspended?

There are three primary forms of secretly suspending users that I know of:

1. A hellbanned user is invisible to all other users, but crucially, not himself. From their perspective, they are participating normally in the community but nobody ever responds to them. They can no longer disrupt the community because they are effectively a ghost. It’s a clever way of enforcing the “don’t feed the troll” rule in the community. When nothing they post ever gets a response, a hellbanned user is likely to get bored or frustrated and leave. I believe it, too; if I learned anything from reading The Great Brain as a child, it’s that the silent treatment is the cruelest punishment of them all.

   I’ve always associated hellbanning with the Something Awful Forums. Per this amazing MetaFilter discussion, it turns out the roots of hellbanning go much deeper – all the way back to an early Telnet BBS system called Citadel, where the “problem user bit” was introduced around 1986. Like so many other things in social software, it keeps getting reinvented over and over again by clueless software developers who believe they’re the first programmer smart enough to figure out how people work. It’s supported in most popular forum and blog software, as documented in the Drupal Cave module.

   (There is one additional form of hellbanning that I feel compelled to mention because it is particularly cruel – when hellbanned users can see only themselves and other hellbanned users. Brrr. I’m pretty sure Dante wrote a chapter about that, somewhere.)

2. A slowbanned user has delays forcibly introduced into every page they visit. From their perspective, your site has just gotten terribly, horribly slow. And stays that way. They can hardly disrupt the community when they’re struggling to get web pages to load. There’s also science behind this one, because per research from Google and Amazon, every page load delay directly reduces participation. Get slow enough, for long enough, and a slowbanned user is likely to seek out greener and speedier pastures elsewhere on the internet.

3. An errorbanned user has errors inserted at random into pages they visit. You might consider this a more severe extension of slowbanning – instead of pages loading slowly, they might not load at all, return cryptic HTTP errors, return the wrong page altogether, fail to load key dependencies like JavaScript and images and CSS, and so forth. I’m sure your devious little brains can imagine dozens of ways things could go “wrong” for an errorbanned user. This one is a bit more esoteric, but it isn’t theoretical; an existing implementation exists in the form of the Drupal Misery module.

Because we try to hew so closely to the real world model of democracy with Stack Exchange, I’m not quite sure how I feel about these sorts of reality-altering tricks that are impossible in the world of atoms. On some level, they feel disingenuous to me. And it’s a bit like wishing users into the cornfield with superhuman powers far beyond the ken of normal people. But I’ve also spent many painful hours trapped in public dialog about users who were, at best, just wasting everyone’s time. Democracy is a wonderful thing, but efficient, it ain’t.

That said, every community is different. I’ve personally talked to people in charge of large online communities – ones you probably participate in every day – and part of the reason those communities haven’t broken down into utter chaos by now is because they secretly hellban and slowban their most problematic users. These solutions do neatly solve the problem of getting troublesome users to “voluntarily” decide to leave a community with a minimum of drama. It’s hard to argue with techniques that are proven to work.

I think everyone has a right to know what sort of jail their community uses, even these secret, invisible ones. But keep in mind that whether it’s timed suspensions, traditional bans, or exotic hellbans and beyond, the goal is the same: civil, sane, and safe online communities for everyone.

Go to Source

HTML5 Speech Input API. Updated icon.

Exciting, eh? Though there was no shortage of hand-wringing over the new icon, of course.

Chrome’s version number has been changing so rapidly lately that every time someone opens a Chrome bug on a Stack Exchange site, I have to check my version against theirs just to make sure we’re still talking about the same software. And once — I swear I am not making this up — the version incremented while I was checking the version.

That was the day I officially stopped caring what version Chrome is. I mean, I care in the sense that sometimes I need to check its dogtags in battle, but as a regular user of Chrome, I no longer think of myself as using a specific version of Chrome, I just … use Chrome. Whatever the latest version is, I have it automagically.

For the longest time, web browsers have been strongly associated with specific versions. The very mention of Internet Explorer 6 or Netscape 4.77 should send a shiver down the spine of any self-respecting geek. And for good reason! Who can forget what a breakout hit Firefox 3 was, or the epochs that Internet Explorer 7, 8 and 9 represent in Microsoft history. But Chrome? Chrome is so fluid that it has transcended software versioning altogether.

This fluidity is difficult to achieve for client software that runs on millions of PCs, Macs, and other devices. Google put an extreme amount of engineering effort into making the Chrome auto-update process “just work”. They’ve optimized the heck out of the update process.

Rather then push put a whole new 10MB update [for each version], we send out a diff that takes the previous version of Google Chrome and generates the new version. We tried several binary diff algorithms and have been using bsdiff up until now. We are big fans of bsdiff – it is small and worked better than anything else we tried.

But bsdiff was still producing diffs that were bigger than we felt were necessary. So we wrote a new diff algorithm that knows more about the kind of data we are pushing – large files containing compiled executables. Here are the sizes for the recent 190.1 -> 190.4 update on the developer channel:

• Full update: 10 megabytes
• bsdiff update: 704 kilobytes
• Courgette update: 78 kilobytes

The small size in combination with Google Chrome’s silent update means we can update as often as necessary to keep users safe.

Google’s Courgette — the French word for Zucchini, oddly enough — is an amazing bit of software optimization, capable of producing uncannily small diffs of binary executables. To achieve this, it has to know intimate details about the source code:

The problem with compiled applications is that even a small source code change causes a disproportional number of byte level changes. When you add a few lines of code, for example, a range check to prevent a buffer overrun, all the subsequent code gets moved to make room for the new instructions. The compiled code is full of internal references where some instruction or datum contains the address (or offset) of another instruction or datum. It only takes a few source changes before almost all of these internal pointers have a different value, and there are a lot of them – roughly half a million in a program the size of chrome.dll.

The source code does not have this problem because all the entities in the source are symbolic. Functions don’t get committed to a specific address until very late in the compilation process, during assembly or linking. If we could step backwards a little and make the internal pointers symbolic again, could we get smaller updates?

Since the version updates are relatively small, they can be downloaded in the background. But even Google hasn’t figured out how to install an update while the browser is running. Yes, there are little alert icons to let you know your browser is out of date, and you eventually do get nagged if you are woefully behind, but updating always requires the browser to restart.

Web applications have it far easier, but they have version delivery problems, too. Consider WordPress, one of the largest and most popular webapps on the planet. We run WordPress on multiple blogs and even have our own WordPress community. WordPress doesn’t auto-update to each new version, but it makes it as painless as I’ve seen for a webapp. Click the update link on the dashboard and WordPress (and its add-ons) update to the latest version all by themselves. There might be the briefest of interruptions in service for visitors to your WordPress site, but then you’re back in business with the latest update.

WordPress needs everyone to update to the latest versions regularly for the same reasons Google Chrome does — security, performance, and stability. An internet full of old, unpatched WordPress or Chrome installations is no less dangerous than an internet full of old, unpatched Windows XP machines.

These are both relatively seamless update processes. But they’re nowhere near as seamless as they should be. One click updates that require notification and restart aren’t good enough. To achieve the infinite version, we software engineers have to go a lot deeper.

Somehow, we have to be able to automatically update software while it is running without interrupting the user at all. Not if — but when — the infinite version arrives, our users probably won’t even know. Or care. And that’s how we’ll know we’ve achieved our goal.

Go to Source

The default, built-in sound chips on most motherboards have evolved from “totally crap” to “surprisingly decent” in the last 5 years. But besides that, in this era of ubiquitous quad core CPUs nearing 4 GHz, it’d be difficult to make a plausible case that you need a discrete set of silicon to handle sound processing, even for the very fanciest of 3D sound algorithms and HRTFs.

That said, if you enjoy music even a little, I still strongly recommend investing in a quality set of headphones. As I wrote in 2005′s Headphone Snobbery:

Am I really advocating spending two hundred dollars on a set of headphones? Yes. Yes I am. Now, you could spend a lot more. This is about extracting the maximum bang for your buck:

1. Unlike your computer, or your car, your headphones will never wear out or become obsolete. I hesitate to say lifetime, but they’re multiple decade investments at the very least.
2. The number one item that affects the music you hear is the speakers. Without a good set of headphones, everything else is irrelevant.
3. The right headphones can deliver sound equivalent to extremely high-end floorstanding speakers worth thousands of dollars.

If you’re the type of person who is perfectly happy listening to 64 kilobit MP3s through a $5 set of beige headphones, that’s fine. There’s nothing wrong with that. Keep on scrolling; this post is not for you.

I realize that there’s a fine line between audiophile and bats**t insane — and that line better not be near any sources of interference! But nice headphones require powerful, reasonably clean output to deliver the best listening experience. This isn’t high end audio crackpot snake oil, it’s actual physics.

I’ll let the guys at headroom explain:

You may have heard of a headphone’s “impedance.” Impedance is the combined resistance and reactivity the headphones present to the amplifier as an electrical load. High impedance cans will usually need more voltage to get up to a solid listening level, so they will often benefit from an amp, especially with portable players that have limited voltage available from their internal batteries. But low impedance cans may require more current, and will lower the damping factor between the amp and headphones. So while low impedance headphones may be driven loud enough from a portable player, the quality of sound may be dramatically improved with an amp.

The size of your headphone will give you some clues to whether an amp may be warranted. Most earbud and in ear headphones are typically very efficient and are less likely to benefit strongly from an amp. Many larger headphones will benefit, or even require, a headphone amp to reach listenable volume levels with portable players.

Thus, once you have a set of nice headphones, you do need some kind of amplified output for them. Something like the Boostaroo, or a Total BitHead. And if you’re on a laptop these outboard solutions might be your only options.

But desktops offer the option of adding a sound card. The good news is that arguably the best sound card on the planet, the Xonar DG, is all of 30 measly bucks. It’s a big step up in fundamental sound quality from even the best current integrated HD audio motherboard sound chips, per this Tech Report review.

It also includes a little something extra of particular interest to us music loving programmers with nice headphones:

Built-in headphone amplification is something you won’t find on a motherboard, but it’s featured in both Xonars. On the DG, Asus has gone with Texas Instruments’ DRV601RTJR, which is optimized for headphone impedances of 32-150 Ω according to the card’s spec sheet. The Xense gets something considerably fancier: a TI amp capable of pushing headphones with impedances up to 600 Ω. Of course, the headphones bundled with the card are rated for an impedance of only 150 Ω. Mid-range stereo cans like Sennheiser’s excellent HD 555s, which we use for listening tests, have a rated impedance of just 50 Ω. You don’t need big numbers for high-quality sound.

The headphone amplification options are a bit buried in the Xonar driver user interface. To get there, select headphone mode, then click the little hammer icon to bring up the headphone amp gain settings.

After my last upgrade, I was truly hoping I could get away with just the on-board Realtek HD audio my motherboard provides. I resisted mightily — but the drop in headphone output quality with the onboard stuff was noticeable. Not to mention that I had to absolutely crank the volume to get even moderate loudness with my fancy-ish Sennheiser HD 600 headphones. The Xonar DG neatly solves both of these problems.

As you probably expected, the answer to the question “Who needs a sound card?” is “Almost nobody.” Except those of us who invested in quality headphones. Rather than spending $30 or $150 on an outboard headphone amp, spend $30 on the Xonar DG to get a substantial sound quality upgrade and a respectable headphone amp to boot.

Go to Source

• The State of Solid State Hard Drives (October 2009)
• Revisiting Solid State Hard Drives (October 2010)

… I feel ethically and morally obligated to let you in on a dirty little secret I’ve discovered in the last two years of full time SSD ownership. Solid state hard drives fail. A lot. And not just any fail. I’m talking about catastrophic, oh-my-God-what-just-happened-to-all-my-data instant gigafail. It’s not pretty.

I bought a set of three Crucial 128 GB SSDs in October 2009 for the original two members of the Stack Overflow team plus myself. As of last month, two out of three of those had failed. And just the other day I was chatting with Joel on the podcast (yep, it’s back), and he casually mentioned to me that the Intel SSD in his Thinkpad, which was purchased roughly around the same time as ours, had also failed.

Portman Wills, friend of the company and generally awesome guy, has a far scarier tale to tell. He got infected with the SSD religion based on my original 2009 blog post, and he went all in. He purchased eight SSDs over the last two years … and all of them failed. The tale of the tape is frankly a little terrifying:

• Super Talent 32 GB SSD, failed after 137 days
• OCZ Vertex 1 250 GB SSD, failed after 512 days
• G.Skill 64 GB SSD, failed after 251 days
• G.Skill 64 GB SSD, failed after 276 days
• Crucial 64 GB SSD, failed after 350 days
• OCZ Agility 60 GB SSD, failed after 72 days
• Intel X25-M 80 GB SSD, failed after 15 days
• Intel X25-M 80 GB SSD, failed after 206 days

You might think after this I’d be swearing off SSDs as unstable, unreliable technology. Particularly since I am the world’s foremost expert on backups.

Well, you’d be wrong. I just went out and bought myself a hot new OCZ Vertex 3 SSD, the clear winner of the latest generation of SSDs to arrive this year. Storage Review calls it the fastest SATA SSD we’ve seen.

Beta firmware or not though, the Vertex 3 is a scorcher. We’ll get into the details later in the review, but our numbers show it as clearly the fastest SATA SSD to hit our bench.

While that shouldn’t be entirely surprising, it’s not just faster like, “Woo, it edged out the prior generation SF-1200 SSDs, yeah!” It’s faster like, “Holy @&#% that’s fast,” boasting 69% faster results in some of our real-world tests.

Solid state hard drives are so freaking amazing performance wise, and the experience you will have with them is so transformative, that I don’t even care if they fail every 12 months on average! I can’t imagine using a computer without a SSD any more; it’d be like going back to dial-up internet or 13″ CRTs or single button mice. Over my dead body, man!

It may seem irrational, but … well, I believe the phenomenon was explained best on the television show How I Met Your Mother by Barney Stinson, a character played brilliantly by geek favorite Neil Patrick Harris:

Barney: There’s no way she’s above the line on the ‘hot/crazy’ scale.

Ted: She’s not even on the ‘hot/crazy’ scale; she’s just hot.

Robin: Wait, ‘hot/crazy’ scale?

Barney: Let me illustrate!

Barney: A girl is allowed to be crazy as long as she is equally hot. Thus, if she’s this crazy, she has to be this hot. You want the girl to be above this line. Also known as the ‘Vickie Mendoza Diagonal’. This girl I dated. She played jump rope with that line. She’d shave her head, then lose 10 pounds. She’d stab me with a fork, then get a boob job. [pause] I should give her a call.

Thing is, SSDs are so scorching hot that I’m willing to put up with their craziness. Consider that just in the last two years, their performance has doubled. Doubled! And the latest, fastest SSDs can even saturate existing SATA interfaces; they need brand new 6 Gbps interfaces to fully strut their stuff. No CPU or memory upgrade can come close to touching that kind of real world performance increase.

Just make sure you have a good backup plan if you’re running on a SSD. I do hope they iron out the reliability kinks in the next 2 generations … but I’ve spent the last two months checking out the hot/crazy solid state drive scale in excruciating detail, and trust me, you want one of these new Vertex 3 SSDs right now.

Go to Source

Late last year, the Netflix Tech Blog wrote about five lessons they learned moving to Amazon Web Services. AWS is, of course, the preeminent provider of so-called “cloud computing”, so this can essentially be read as key advice for any website considering a move to the cloud. And it’s great advice, too. Here’s the one bit that struck me as most essential:

We’ve sometimes referred to the Netflix software architecture in AWS as our Rambo Architecture. Each system has to be able to succeed, no matter what, even all on its own. We’re designing each distributed system to expect and tolerate failure from other systems on which it depends.

If our recommendations system is down, we degrade the quality of our responses to our customers, but we still respond. We’ll show popular titles instead of personalized picks. If our search system is intolerably slow, streaming should still work perfectly fine.

One of the first systems our engineers built in AWS is called the Chaos Monkey. The Chaos Monkey’s job is to randomly kill instances and services within our architecture. If we aren’t constantly testing our ability to succeed despite failure, then it isn’t likely to work when it matters most – in the event of an unexpected outage.

Which, let’s face it, seems like insane advice at first glance. I’m not sure many companies even understand why this would be a good idea, much less have the guts to attempt it. Raise your hand if where you work, someone deployed a daemon or service that randomly kills servers and processes in your server farm.

Now raise your other hand if that person is still employed by your company.

Who in their right mind would willingly choose to work with a Chaos Monkey?

Sometimes you don’t get a choice; the Chaos Monkey chooses you. At Stack Exchange, we struggled for months with a bizarre problem. Every few days, one of the servers in the Oregon web farm would simply stop responding to all external network requests. No reason, no rationale, and no recovery except for a slow, excruciating shutdown sequence requiring the server to bluescreen before it would reboot.

We spent months — literally months — chasing this problem down. We walked the list of everything we could think of to solve it, and then some:

• swapping network ports
• replacing network cables
• a different switch
• multiple versions of the network driver
• tweaking OS and driver level network settings
• simplifying our network configuration and removing TProxy for more traditional X-FORWARDED-FOR
• switching virtualization providers
• changing our TCP/IP host model
• getting Kernel hotfixes and applying them
• involving high-level vendor support teams
• some other stuff that I’ve now forgotten because I blacked out from the pain

At one point in this saga our team almost came to blows because we were so frustrated. (Well, as close to “blows” as a remote team can get over Skype, but you know what I mean.) Can you blame us? Every few days, one of our servers — no telling which one — would randomly wink off the network. The Chaos Monkey strikes again!

Even in our time of greatest frustration, I realized that there was a positive side to all this:

• Where we had one server performing an essential function, we switched to two.
• If we didn’t have a sensible fallback for something, we created one.
• We removed dependencies all over the place, paring down to the absolute minimum we required to run.
• We implemented workarounds to stay running at all times, even when services we previously considered essential were suddenly no longer available.

Every week that went by, we made our system a tiny bit more redundant, because we had to. Despite the ongoing pain, it became clear that Chaos Monkey was actually doing us a big favor by forcing us to become extremely resilient. Not tomorrow, not someday, not at some indeterminate “we’ll get to it eventually” point in the future, but right now where it hurts.

Now, none of this is new news; our problem is long since solved, and the Netflix Tech Blog article I’m referring to was posted last year. I’ve been meaning to write about it, but I’ve been a little busy. Maybe the timing is prophetic; AWS had a huge multi-day outage last week, which took several major websites down, along with a constellation of smaller sites.

Notably absent from that list of affected AWS sites? Netflix.

When you work with the Chaos Monkey, you quickly learn that everything happens for a reason. Except for those things which happen completely randomly. And that’s why, even though it sounds crazy, the best way to avoid failure is to fail constantly.

Go to Source

My old low-power Athlon X2 based HTPC generally worked great, but still struggled with some occasional 1080p content. And when you have a toddler in the house, believe me, you need reliable 1080p playback. Only the finest in children’s entertainment for my spawned process, I say!

When I recently had to transcode Megamind down to 720p to get it to play back without stuttering or pausing at times… I knew my current HTPC’s days were numbered.

(Megamind is hilarious and highly recommended, by the way; it’s far better than its Metacritic and Rotten Tomatoes percentages would seem to indicate.)

Now that Intel has finally released their Sandy Bridge CPUs — the first with integrated GPUs — I was eager to revisit and rebuild. The low power Core i3-2100T is the one I had my eye on, with a miserly TDP of 35 watts. Combine that with a decent Mini-ITX motherboard and a few other essential parts, and you’re good to go:

Now, I am fudging a bit here. This is just the basic level of hardware to get a functional home theater PC. I didn’t actually buy a case, PSU, or even hard drive for that matter; I recycled many of my old existing parts, so my personal outlay was all of 300 bucks. I’m including the fuller part list as courtesy recommendations in case you’re starting from scratch. You also might want to add a Blu-Ray drive, and perhaps a Windows 7 Home Premium license ($99) for its excellent 10-foot Windows Media Center interface.

The magical part here is the extreme level of hardware integration: the CPU has a GPU and memory controller on die, and the motherboard has optical digital out and HDMI out built in. It’s delightfully simple to build and downright cheap. Just assemble it, install your OS of choice (sorry, Apple fans), then plug it into your receiver and television and boot it up.

My results? I’ll just get right to the good part, but please bear in mind each step is about twice as powerful as the one before:

I know I get way too excited about this stuff, but … holy crap, 22 tesla-lovin’ watts at idle!

The kill-a-watt never lies. To be fair, it’s more like 25 watts idle with torrents in the background. This little box is remarkably efficient; even when playing back a 1080p video it’s not unusual to see CPU usage well under 50%, which equates to around 30-35 watts in practice. Under full, artificial multithreaded Prime95 load, it tops out at an absolute peak of 55 watts.

This is a killer setup, but don’t take my word for it. There is an excruciatingly in-depth review of essentially the same system at Missing Remote, with a particular eye toward home theater duties. Spoiler: they loved the hell out of it too. And it compromises almost nothing in performance, with a Windows Experience score of 5.1 — that would be a solid 5.8 if you factored out desktop Aero performance.

(Also, in case you’re wondering, I intentionally dropped the analog cable tuner. All modern cable is now digital, which means awkward DRM-ed up the wazoo CableCard systems. I’ve cancelled cable altogether; I’d rather take that $60+ per month and use it to support innovative companies who will deliver media through the internet, like Netflix, Hulu, etcetera. Or as I like to call it: the future, unless the media congolomerates with vaults full of cash manage to subvert net neutrality.)

When all is said and done, I have a new always-on, does-anything home theater box that is twice as fast as the one I built in 2008, while consuming less than half the power.

I’ve been a computer nerd since age 8, and I just turned 40. I should be jaded by computer hardware pornography by now, but I still find this progress amazing. At this rate, I can’t wait to find out what my 2014 home theater PC will look like.

Go to Source

My old low-power Athlon X2 based HTPC generally worked great, but still struggled with some occasional 1080p content. And when you have a toddler in the house, believe me, you need reliable 1080p playback. Only the finest in children’s entertainment for my spawned process, I say!

When I recently had to transcode Megamind down to 720p to get it to play back without stuttering or pausing at times… I knew my current HTPC’s days were numbered.

(Megamind is hilarious and highly recommended, by the way; it’s far better than its Metacritic and Rotten Tomatoes percentages would seem to indicate.)

Now that Intel has finally released their Sandy Bridge CPUs — the first with integrated GPUs — I was eager to revisit and rebuild. The low power Core i3-2100T is the one I had my eye on, with a miserly TDP of 35 watts. Combine that with a decent Mini-ITX motherboard and a few other essential parts, and you’re good to go:

Now, I am fudging a bit here. This is just the basic level of hardware to get a functional home theater PC. I didn’t actually buy a case, PSU, or even hard drive for that matter; I recycled many of my old existing parts, so my personal outlay was all of 300 bucks. I’m including the fuller part list as courtesy recommendations in case you’re starting from scratch. You also might want to add a Blu-Ray drive, and perhaps a Windows 7 Home Premium license ($99) for its excellent 10-foot Windows Media Center interface.

The magical part here is the extreme level of hardware integration: the CPU has a GPU and memory controller on die, and the motherboard has optical digital out and HDMI out built in. It’s delightfully simple to build and downright cheap. Just assemble it, install your OS of choice (sorry, Apple fans), then plug it into your receiver and television and boot it up.

My results? I’ll just get right to the good part, but please bear in mind each step is about twice as powerful as the one before:

I know I get way too excited about this stuff, but … holy crap, 22 tesla-lovin’ watts at idle!

The kill-a-watt never lies. To be fair, it’s more like 25 watts idle with torrents in the background. This little box is remarkably efficient; even when playing back a 1080p video it’s not unusual to see CPU usage well under 50%, which equates to around 30-35 watts in practice. Under full, artificial multithreaded Prime95 load, it tops out at an absolute peak of 55 watts.

This is a killer setup, but don’t take my word for it. There is an excruciatingly in-depth review of essentially the same system at Missing Remote, with a particular eye toward home theater duties. Spoiler: they loved the hell out of it too. And it compromises almost nothing in performance, with a Windows Experience score of 5.1 — that would be a solid 5.8 if you factored out desktop Aero performance.

(Also, in case you’re wondering, I intentionally dropped the analog cable tuner. All modern cable is now digital, which means awkward DRM-ed up the wazoo CableCard systems. I’ve cancelled cable altogether; I’d rather take that $60+ per month and use it to support innovative companies who will deliver media through the internet, like Netflix, Hulu, etcetera. Or as I like to call it: the future, unless the media congolomerates with vaults full of cash manage to subvert net neutrality.)

When all is said and done, I have a new always-on, does-anything home theater box that is twice as fast as the one I built in 2008, while consuming less than half the power.

I’ve been a computer nerd since age 8, and I just turned 40. I should be jaded by computer hardware pornography by now, but I still find this progress amazing. At this rate, I can’t wait to find out what my 2014 home theater PC will look like.

Go to Source

Before you trot out the pitchforks and torches, let me explain.

Over the last 6 years, I’ve come to believe deeply in the idea that that becoming a great programmer has very little to do with programming. Yes, it takes a modicum of technical skill and dogged persistence, absolutely. But even more than that, it takes serious communication skills:

The difference between a tolerable programmer and a great programmer is not how many programming languages they know, and it’s not whether they prefer Python or Java. It’s whether they can communicate their ideas. By persuading other people, they get leverage. By writing clear comments and technical specs, they let other programmers understand their code, which means other programmers can use and work with their code instead of rewriting it. Absent this, their code is worthless.

That is of course a quote from my co-founder Joel Spolsky, and it’s one of my favorites.

In defense of my fellow programmers, communication with other human beings is not exactly what we signed up for. We didn’t launch our careers in software development because we loved chatting with folks. Communication is just plain hard, particularly written communication. How exactly do you get better at something you self-selected out of? Blogging is one way:

People spend their entire lives learning how to write effectively. It isn’t something you can fake. It isn’t something you can buy. You have to work at it.

That’s exactly why people who are afraid they can’t write should be blogging.

It’s exercise. No matter how out of shape you are, if you exercise a few times a week, you’re bound to get fitter. Write a small blog entry a few times every week and you’re bound to become a better writer. If you’re not writing because you’re intimidated by writing, well, you’re likely to stay that way forever.

Even with the best of intentions, telling someone “you should blog!” never works. I know this from painful first hand experience. Blogging isn’t for everyone. Even a small blog entry can seem like an insurmountable, impenetrable, arbitrary chunk of writing to the average programmer. How do I get my fellow programmers to blog without blogging, to write without writing?

By cheating like hell, that’s how.

Consider this letter I received:

I’m not sure if you have thought about this side effect or not, but Stack Overflow has taught me more about writing effectively than any class I’ve taken, book I’ve read, or any other experience I have had before.

I can think of no other medium where I can test my writing chops (by writing an answer), get immediate feedback on its quality (particularly when writing quality trumps technical correctness, such as subjective questions) and see other peoples attempts as well and how they compare with mine. Votes don’t lie and it gives me a good indicator of how well an email I might send out to future co-workers would be received or a business proposal I might write.

Over the course of the past 5 months all the answers I’ve been writing have been more and more refined in terms of the quality. If I don’t end up as the top answer I look at the answer that did and study what they did differently and where I faltered. Was I too verbose or was I too terse? Was I missing the crux of the question or did I hit it dead on?

I know that you said that writing your Coding Horror blog helped you greatly in refining your writing over the years. Stack Overflow has been doing the same for me and I just wanted to thank you for the opportunity. I’ve decided to setup a coding blog in your footsteps and I just registered a domain today. Hopefully that will go as well as writing on SO has. There are no tougher critics than fellow programmers who scrutinize every detail, every technical remark and grammar structure looking for mistakes. If you can effectively write for and be accepted by a group of programmers you can write for anyone.

Joel and I have always positioned Stack Overflow, and all the other Stack Exchange Q&A sites, as lightweight, focused, “fun size” units of writing.

Yes, by God, we will trick you into becoming a better writer if that’s what it takes – and it always does. Stack Overflow has many overtly gamelike elements, but it is a game in service of the greater good – to make the internet better, and more importantly, to make you better. Seeing my fellow programmers naturally improve their written communication skills while participating in a focused, expert Q&A community with their peers? Nothing makes me prouder.

Beyond programming, there’s a whole other community of peers out there who grok how important writing is, and will support you in sharpening your saw, er, pen. We have our own, too.

If you’re an author, editor, reviewer, blogger, copywriter, or aspiring writer of any kind, professional or otherwise – check out writers.stackexchange.com. Becoming a more effective writer is the one bedrock skill that will further your professional career, no matter what you choose to do.

But mostly, you should write. I thought Jon Skeet summed it up particularly well here:

Everyone should write a lot – whether it’s a blog, a book, Stack Overflow answers, emails or whatever. Write, and take some care over it. Clarifying your communication helps you to clarify your own internal thought processes, in my experience. It’s amazing how much you find you don’t know when you try to explain something in detail to someone else. It can start a whole new process of discovery.

The process of writing is indeed a journey of discovery, one that will last the rest of your life. It doesn’t ultimately matter whether you’re writing a novel, a printer review, a Stack Overflow answer, fan fiction, a blog entry, a comment, a technical whitepaper, some emo LiveJournal entry, or even meta-talk about writing itself. Just get out there and write!

Go to Source

There’s plenty of evidence that we’re entering another tech bubble. It’s just less visible to people outside the tech industry because there’s no corresponding stock market IPO frenzy. Yet.

There are two films which captured the hyperbole and excess of the original dot com bubble especially well.

The first is the documentary Startup.com. It’s about the prototypical web 1.0 company: one predicated on an idea that made absolutely no sense, which proceeded to flame out in a spectacular and all too typical way for the era. This one just happened to occur on digital film. The govworks.com website described in the documentary, the one that burned through $60 million in 18 months, is now one of those ubiquitous domain squatter pages. A sign of the times, perhaps.

The second film was one I had always wanted to see, but wasn’t able to until a few days ago: Code Rush. For a very long time, Code Rush was almost impossible to find, but the activism of Andy Baio nudged the director to make the film available under Creative Commons. You can now watch it online — and you absolutely should.

Remember when people charged money for a web browser? That was Netscape.

Code Rush is a PBS documentary recorded at Netscape from 1998 – 1999, focusing on the open sourcing of the Netscape code. As the documentary makes painfully clear, this wasn’t an act of strategy so much as an act of desperation. That’s what happens when the company behind the world’s most ubiquitous operating system decides a web browser should be a standard part of the operating system.

Everyone in the documentary knows they’re doomed; in fact, the phrase “we’re doomed” is a common refrain throughout the film. But despite the gallows humor and the dark tone, parts of it are oddly inspiring. These are engineers who are working heroic, impossible schedules for a goal they’re not sure they can achieve — or that they’ll even survive as an organization long enough to even finish.

The most vivid difference between Startup.com and Code Rush is that Netscape, despite all their other mistakes and missteps, didn’t just burn through millions of dollars for no discernable reason. They produced a meaningful legacy:

• Through Netscape Navigator, the original popularization of HTML and the internet itself.
• With the release of the Netscape source code on March 31st, 1998, the unlikely birth of the commercial open source movement.
• Eventually producing the first credible threat to Internet Explorer in the form of Mozilla Firefox 1.0 in 2004.

Do you want money? Fame? Job security? Or do you want to change the world … eventually? Consider how many legendary hackers went on to brilliant careers from Netscape: Jamie Zawinski, Brendan Eich, Stuart Parmenter, Marc Andreeseen. The lessons of Netscape live on, even though the company doesn’t. Code Rush is ultimately a meditation on the meaning of work as a programmer.

I’d like to think that when Facebook – the next Google and Microsoft rolled into one – goes public in early 2012, the markets will react rationally. More likely, people will all collectively lose their damn minds again and we’ll be thrust into a newer, bigger, even more insane tech bubble than the first one.

Yes, you will have incredibly lucrative job offers in this bubble. That’s the easy part. As Startup.com and Code Rush illustrate, the hard part is figuring out why you are working all those long hours. Consider carefully, lest the arc of your career mirror that of so many failed tech bubble companies: lived fast, died young, left a tired corpse.

Go to Source

640K [of computer memory] ought to be enough for anybody. — Bill Gates

It’s amusing, but Bill Gates never actually said that:

I’ve said some stupid things and some wrong things, but not that. No one involved in computers would ever say that a certain amount of memory is enough for all time … I keep bumping into that silly quotation attributed to me that says 640K of memory is enough. There’s never a citation; the quotation just floats like a rumor, repeated again and again.

One of the few killer features of the otherwise unexciting Intel Core i7 platform upgrade* is the subtle fact that Core i7 chips use triple channel memory. That means three memory slots at a minimum, and in practice most Core i7 motherboards have six memory slots.

The price of DDR3 ram has declined to the point that populating all six slots of memory with 4 GB memory is, well, not cheap — but quite attainable at $299 and declining.

Twenty-four gigabytes of system memory for a mere $299! That’s about $12.50 per gigabyte.

(And if you don’t have a Core i7 system, they’re not expensive to build, either. You can pair an inexpensive motherboard with even the slowest and cheapest triple channel compatible i7-950, which is plenty speedy – and overclocks well, if you’re into that. Throw in the 24 GB of ram, and it all adds up to about $800 total. Don’t forget the power supply and CPU cooler, though.)

Remember when one gigabyte of system memory was considered a lot? For context, our first “real” Stack Overflow database server had 24 GB of memory. Now I have that much in my desktop … just because I can. Well, that’s not entirely true, as we do work with some sizable databases while building the Stack Exchange network.

I guess having 24 gigabytes of system memory is a little extravagant, but at these prices — why not? What’s the harm in having obscene amounts of memory, making my system effectively future proof?

I have to say that in 1981, making those decisions, I felt like I was providing enough freedom for 10 years. That is, a move from 64k to 640k felt like something that would last a great deal of time. Well, it didn’t – it took about only 6 years before people started to see that as a real problem. — Bill Gates

To me, it’s more about no longer needing to think about memory as a scarce resource, something you allocate carefully and manage with great care. There’s just .. lots. As Clay Shirky once related to me, via one of his college computer science professors:

Algorithms are for people who don’t know how to buy RAM.

I mean, 24 GB of memory should be enough for anybody… right?

* it’s only blah on the desktop; on the server the Nehalem architecture is indeed a monster and anyone running a server should upgrade to it, stat.

Go to Source

When 88.2% of all traffic for your website comes from a single source, criticizing that single source feels … risky. And perhaps a bit churlish, like looking a gift horse in the mouth, or saying something derogatory in public about your Valued Business Partner^tm.

Still, looking at the statistics, it’s hard to avoid the obvious conclusion. I’ve been told many times that Google isn’t a monopoly, but they apparently play one on the internet. You are perfectly free to switch to whichever non-viable alternative web search engine you want at any time. Just breathe in that sweet freedom, folks.

Sarcasm aside, I greatly admire Google. My goal is not to be acquired, because I’m in this thing for the long haul – but if I had to pick a company to be acquired by, it would probably be Google. I feel their emphasis on the information graph over the social graph aligns more closely with our mission than almost any other potential suitor I can think of. Anyway, we’ve been perfectly happy with Google as our de-facto traffic sugar daddy since the beginning. But last year, something strange happened: the content syndicators began to regularly outrank us in Google for our own content.

Syndicating our content is not a problem. In fact, it’s encouraged. It would be deeply unfair of us to assert ownership over the content so generously contributed to our sites and create an underclass of digital sharecroppers. Anything posted to Stack Overflow, or any Stack Exchange Network site for that matter, is licensed back to the community in perpetuity under Creative Commons cc-by-sa. The community owns their contributions. We want the whole world to teach each other and learn from the questions and answers posted on our sites. Remix, reuse, share – and teach your peers! That’s our mission. That’s why I get up in the morning.

However, implicit in this strategy was the assumption that we, as the canonical source for the original questions and answers, would always rank first. Consider Wikipedia – when was the last time you clicked through to a page that was nothing more than a legally copied, properly attributed Wikipedia entry encrusted in advertisements? Never, right? But it is in theory a completely valid, albeit dumb, business model. That’s why Joel Spolsky and I were confident in sharing content back to the community with almost no reservations – because Google mercilessly penalizes sites that attempt to game the system by unfairly profiting on copied content. Remixing and reusing is fine, but mass-producing cheap copies encrusted with ads … isn’t.

I think of this as common sense, but it’s also spelled out explicitly in Google’s webmaster content guidelines.

However, some webmasters attempt to improve their page’s ranking and attract visitors by creating pages with many words but little or no authentic content. Google will take action against domains that try to rank more highly by just showing scraped or other auto-generated pages that don’t add any value to users. Examples include:

Scraped content. Some webmasters make use of content taken from other, more reputable sites on the assumption that increasing the volume of web pages with random, irrelevant content is a good long-term strategy. Purely scraped content, even from high-quality sources, may not provide any added value to your users without additional useful services or content provided by your site. It’s worthwhile to take the time to create original content that sets your site apart. This will keep your visitors coming back and will provide useful search results.

In 2010, our mailboxes suddenly started overflowing with complaints from users – complaints that they were doing perfectly reasonable Google searches, and ending up on scraper sites that mirrored Stack Overflow content with added advertisements. Even worse, in some cases, the original Stack Overflow question was nowhere to be found in the search results! That’s particularly odd because our attribution terms require linking directly back to us, the canonical source for the question, without nofollow. Google, in indexing the scraped page, cannot avoid seeing that the scraped page links back to the canonical source. This culminated in, of all things, a special browser plug-in that redirects to Stack Overflow from the ripoff sites. How totally depressing. Joel and I thought this was impossible. And I felt like I had personally failed all of you.

The idea that there could be something wrong with Google was inconceivable to me. Google is gravity on the web, an omnipresent constant; blaming Google would be like blaming gravity for my own clumsiness. It wasn’t even an option. I started with the golden rule: it’s always my fault. We did a ton of due diligence on webmasters.stackexchange.com to ensure we weren’t doing anything overtly stupid, and uber-mensch Matt Cutts went out of his way to investigate the hand-vetted search examples contributed in response to my tweet asking for search terms where the scrapers dominated. Issues were found on both sides, and changes were made. Success!

Despite the semi-positive resolution, I was disturbed. If these dime-store scrapers were doing so well and generating so much traffic on the back of our content – how was the rest of the web faring? My enduring faith in the gravitational constant of Google had been shaken. Shaken to the very core.

Throughout my investigation I had nagging doubts that we were seeing serious cracks in the algorithmic search foundations of the house that Google built. But I was afraid to write an article about it for fear I’d be claimed an incompetent kook. I wasn’t comfortable sharing that opinion widely, because we might be doing something obviously wrong. Which we tend to do frequently and often. Gravity can’t be wrong. We’re just clumsy … right?

I can’t help noticing that we’re not the only site to have serious problems with Google search results in the last few months. In fact, the drum beat of deteriorating Google search quality has been practically deafening of late:

• Why We Desperately Need a New (and Better) Google
• Dishwashers, and How Google Eats Its Own Tail
• Content Farms: Why Media, Blogs & Google Should Be Worried
• On the increasing uselessness of Google
• Google, Google, Why Hast Thou Forsaken the Manolo?

Anecdotally, my personal search results have also been noticeably worse lately. As part of Christmas shopping for my wife, I searched for “iPhone 4 case” in Google. I had to give up completely on the first two pages of search results as utterly useless, and searched Amazon instead.

People whose opinions I respect have all been echoing the same sentiment — Google, the once essential tool, is somehow losing its edge. The spammers, scrapers, and SEO’ed-to-the-hilt content farms are winning.

Like any sane person, I’m rooting for Google in this battle, and I’d love nothing more than for Google to tweak a few algorithmic knobs and make this entire blog entry moot. Still, this is the first time since 2000 that I can recall Google search quality ever declining, and it has inspired some rather heretical thoughts in me — are we seeing the first signs that algorithmic search has failed as a strategy? Is the next generation of search destined to be less algorithmic and more social?

It’s a scary thing to even entertain, but maybe gravity really is broken.

Go to Source

This weekend we discovered that Gawker Media’s servers were compromised, resulting in a security breach at Lifehacker, Gizmodo, Gawker, Jezebel, io9, Jalopnik, Kotaku, Deadspin, and Fleshbot. If you’re a commenter on any of our sites, you probably have several questions.

It’s no Black Sunday or iPod modem firmware hack, but it has release notes — and the story it tells is as epic as Beowulf:

So, here we are again with a monster release of ownage and data droppage. Previous attacks against the target were mocked, so we came along and raised the bar a little. How’s this for “script kids”? Your empire has been compromised, your servers, your databases, online accounts and source code have all been ripped to shreds!

You wanted attention, well guess what, You’ve got it now!

Read those release notes. It’ll explain how the compromise unfolded, blow by blow, from the inside.

Gawker is operated by Nick Denton, notorious for the unapologetic and often unethical “publish whatever it takes to get traffic” methods endorsed on his network. Do you remember the iPhone 4 leak? That was Gawker. Do you remember the article about bloggers being treated as virtual sweatshop workers? That was Gawker. Do you remember hearing about a blog lawsuit? That was probably Gawker, too.

Some might say having every account on your network compromised is exactly the kind of unwanted publicity attention that Gawker was founded on.

Personally, I’m more interested in how we can learn from this hack. Where did Gawker go wrong, and how can we avoid making those mistakes on our projects?

1. Gawker saved passwords. You should never, ever store user passwords. If you do, you’re storing passwords incorrectly. Always store the salted hash of the password — never the password itself! It’s so easy, even members of Mensa er .. can’t .. figure it out.
2. Gawker used encryption incorrectly. The odd choice of archaic DES encryption meant that the passwords they saved were all truncated to 8 characters. No matter how long your password actually was, you only had to enter the first 8 characters for it to work. So much for choosing a secure pass phrase. Encryption is only as effective as the person using it. I’m not smart enough to use encryption, either, as you can see in Why Isn’t My Encryption.. Encrypting?
3. Gawker asked users to create a username and password on their site. The FAQ they posted about the breach has two interesting clarifications:
   

   2) What if I logged in using Facebook Connect? Was my password compromised?
   No. We never stored passwords of users who logged in using Facebook Connect.

   3) What if I linked my Twitter account with my Gawker Media account? Was my Twitter password compromised?
   No. We never stored Twitter passwords from users who linked their Twitter accounts with their Gawker Media account.

   That’s right, people who used their internet driver’s license to authenticate on these sites had no security problems at all! Does the need to post a comment on Gizmodo really justify polluting the world with yet another username and password? It’s only the poor users who decided to entrust Gawker with a unique username and ‘secure’ password who got compromised.

(Beyond that, “don’t be a jerk” is good advice to follow in business as well as your personal life. I find that you generally get back what you give. When your corporate mission is to succeed by exploiting every quasi-legal trick in the book, surely you can’t be surprised when you get the same treatment in return.)

But honestly, as much as we can point and laugh at Gawker and blame them for this debacle, there is absolutely nothing unique or surprising about any of this. Regular readers of my blog are probably bored out of their minds by now because I just trotted out a whole bunch of blog posts I wrote 3 years ago. Again.

Here’s the dirty truth about website passwords: the internet is full of websites exactly like the Gawker network. Let’s say you have good old traditional username and passwords on 50 different websites. That’s 50 different programmers who all have different ideas of how your password should be stored. I hope for your sake you used a different (and extremely secure) password on every single one of those websites. Because statistically speaking, you’re screwed.

In other words, the more web sites you visit, the more networks you touch and trust with a username and password combination — the greater the odds that at least one of those networks will be compromised exactly like Gawker was, and give up your credentials for the world to see. At that point, unless you picked a strong, unique password on every single site you’ve ever visited, the situation gets ugly.

The bad news is that most users don’t pick strong passwords. This has been proven time and time again, and the Gawker data is no different. Even worse, most users re-use these bad passwords across multiple websites. That’s why this ugly Twitter worm suddenly appeared on the back of a bunch of compromised Gawker accounts.

Now do you understand why I’ve been so aggressive about promoting the concept of the internet driver’s license? That is, logging on to a web site using a set of third party credentials from a company you can actually trust to not be utterly incompetent at security? Sure, we’re centralizing risk here to, say, Google, or Facebook — but I trust Google a heck of a lot more than I trust J. Random Website, and this really is no different in practice than having password recovery emails sent to your GMail account.

I’m not here to criticize Gawker. On the contrary, I’d like to thank them for illustrating in broad, bold relief the dirty truth about website passwords: we’re all better off without them. If you’d like to see a future web free of Gawker style password compromises — stop trusting every random internet site with a unique username and password! Demand that they allow you to use your internet driver’s license — that is, your existing Twitter, Facebook, Google, or OpenID credentials — to log into their website.

Go to Source

But then maybe I shouldn’t be so surprised. I’ve talked about the relationship between gamer and programmer before:

• Programming Games, Analyzing Games
• Everything I Needed to Know About Programming I Learned from BASIC
• Game Player, Game Programmer

I used to recommend games on this very blog that I particularly enjoyed and felt were worthy of everyone’s attention. I don’t do this a lot any more, now that my blogging schedule has slipped to one post a week, if I’m lucky. (If you’re wondering why, it’s because running your own business is crazy stupid amounts of work when you turn it up to eleven.) Here are a few games I’ve recommended in the past:

• Guitar Hero: Are You Ready to Rock?
• Darwinia
• DEFCON: Shall We Play a Game?
• Company of Heroes
• Living the Dream: Rock Band
• Feeding my Graphics Card Addiction (Fallout 3)
• My Software is Being Pirated (World of Goo)

I haven’t had a ton of time to play games, other than the inevitable Rock Band 3, but I’ve been consumed by another game I had no idea would become so addictive — Just Cause 2.

It’s what you might call an open world sandbox game, in the vein of the Grand Theft Autos. But I could never get into the GTA games, even after trying GTA 3 and its sequels Vice City and San Andreas. They just left me cold, somehow.

Where GTA and its ilk often felt a tad too much like work for my tastes, Just Cause 2 is almost the opposite — it is non-stop, full blown open world pandemonium from start to finish. One of the game’s explicit goals is that you advance the plot by blowing stuff up. No, seriously. I’m not kidding. You have an entire 1000+ square kilometer island paradise at your disposal, filled with cities and military bases, spanning the range from snowy mountains to deserts to idyllic beaches — all just waiting for you to turn them into “chaos points” … by any means necessary.

Of course, you get around by hijacking whatever vehicles happen by, be they boats, airplanes, jumbo jets, cars, tanks, trucks, buses, monster trucks, motorcycles, scooters, tractors or anything in between. Even on foot it is fun to navigate the island of Panau, because the developers gave us an impossibly powerful personal zipline that you can fire at any object in the game to propel yourself toward it. Combine that with the magical parachute you can deploy anywhere, anytime, and they make for some fascinating diversions (parasailing anyone?). You can also use the zipline to attach any two objects together. Think about that for a second. Have you ever wondered what happens when you zipline a moving vehicle to a tree? Or a pedestrian? Or another vehicle? Hmm. As a result, simply going walkabout on the island is more fun than I ever would have imagined.

Between the 49 plot missions, 9 stronghold takeovers, 104 unique vehicles, the optional boat/plane/car/parachute race missions,the opportunities for insane stunt points, the umpteen zillion upgrade crates and faction objects to collect, and the 360+ locations in the 1000+ square kilometers of Panau — there’s always something interesting happening around every corner. And whatever it is, it’s probably beautiful and blows up real good.

In short, Just Cause 2 is deliriously, stupidly, absurdly entertaining. I can’t even remember the last game I completed where I felt compelled to go back after finishing the main storyline to discover even more areas I missed during my initial playthrough and get (most of) the in-game achievements. Whatever amount of time you have to play, Just Cause 2 will happily fill it with totally unscripted, awesome open world pandemonium.

Don’t take my word for it; even the notoriously acidic game reviewer Yahtzee had almost nothing negative to say about Just Cause 2, which is his version of a positive review. And Metacritic gives Just Cause 2 a solid 84. Not that it can’t be improved, of course; after such a sublime sandbox experience, I’m desperately curious to see what they’ll add for Just Cause 3.

Luckily for you, the game has been out long enough that it can be picked up for a song on PS3, Xbox, or PC. Steam has Just Cause 2 on sale right now in an 8 player pack for $60, and Amazon has all versions in stock for under $30. Beware, though, as the PC version does require a pretty solid video card along with Windows Vista or newer — but the upside is that I have mine cranked up to 2048×1152 with almost all the options on, and it rarely dips below 60 fps.

I spent my holidays on the beautiful island of Panau, and I don’t regret a second of it. If you’re looking for a vacation spot, I heartily recommend the open world sandbox of Panau. But while you’re visiting, do be mindful of any errant gunfire, vehicles, and explosions.

Go to Source

I realize that OpenID is far from an ideal solution. But right now, the one-login-per-website problem is so bad that I am willing to accept these tradeoffs for a partial worse is better solution. There’s absolutely no way I’d put my banking credentials behind an OpenID. But there are also dozens of sites that I don’t need anything remotely approaching banking-grade security for, and I use these sites far more often than my bank. The collective pain of remembering all these logins — and the way my email inbox becomes a de-facto collecting point and security gateway for all of them — is substantial.

It always pained me greatly that every rinky-dink website on the entire internet demanded that I create a special username and password just for them. Yes, if you’re an alpha geek, then you probably use a combination of special software and USB key from your utility belt to generate secure usernames and passwords for the dozens of websites you frequent. But for the vast, silent majority of normals, who know nothing of security but desire convenience above all, this means one thing: using the same username and password over and over. And it’s probably a simple password, too.

This is the status quo of identity on the internet. It is deeply and fundamentally broken.

But it doesn’t have to be this way. If you open your wallet (or purse, or man-purse, or whatever), I bet you’ll find a variety of credentials you use to prove your identity wherever you go.

The average wallet contains a few different forms of identity with varying strengths:

• Strong: California driver’s license, student ID
• Moderate: credit cards, health insurance card, video rental membership, gym card
• Weak: Albertson’s Preferred Card, Best Buy Rewards Zone Card, Coffee loyalty card

(and sometimes even, uh, cards for free lapdances, apparently)

In the real world, we don’t regularly hold two dozen forms of identity like we expect people to on the web. Not only would you be carrying around the freaking Constanza wallet at that point, it would be insane. In the real world, we somehow manage to get by with about two or three strong forms of identity, complemented by a few other weaker forms to taste.

I’m proposing that our web wallets begin to mimic our physical wallets. Whenever a website needs to know who I am, they should ask to see my Internet Driver’s License.

Now, I don’t literally mean a driver’s license. I’m using this term figuratively to mean online credentials that I can re-use in more than one place on the internet. If all I want to do is leave a comment on a blog — like, say, this one — then one of the weaker forms of identity will surely do. If I’m starting a new bank account, or setting up a profile on a dating website, then maybe a stronger credential from my virtual wallet is necessary.

The core concept that users need to get used to is logging in to a website by showing a third party credential to validate their identity. This idea isn’t nearly as crazy as it seemed in 2008. How many websites can you log into by showing your Facebook, Google, or Twitter credentials now? Lots!

The whole online identity situation may seem as impossible as peace in the Middle East at this point. But when faced with a problem that appears intractable, is your solution to throw your hands up, mindlessly embrace the status quo, and wearily sigh “whaddaya gonna do?”

Some people do that. It’s their right. Personally, I prefer to be the change I want to see. So for us, on Stack Overflow and the Stack Exchange network, that means aggressively promoting the concept of the Internet Driver’s License. Including educating users as necessary.

For example, consider this ATM machine. To use it, do I need to sign up for an account at Shanghai Peking Development Bank? No. I can use any form of trusted third-party credentials the machine supports.

Similarly, to log into any Stack Exchange site, including Stack Overflow, present any OpenID or OAuth 2.0 compliant identity provider as your Internet Driver’s License.

When we founded Stack Overflow, we set out with the explicit mission to make the internet better. Adding yet another meaningless username and password to the fabric of the web does not make it better. What does make the internet better is continued pursuit of better, simpler, re-usable forms of third party online identity. That’s why I urge you to join me in supporting OpenID, OAuth 2.0, and any other promising implementations of the Internet Driver’s License.

Go to Source

• Connect to a public, unencrypted WiFi network. In other words, a WiFi network that doesn’t require a password before you can connect to it.
• Install Firefox and the Firesheep add-in.
• Wait. Maybe have a latte while you’re waiting.
• Click on the user / website icons that appear over time in Firesheep to instantly log in as that user on that website.

Crazy! This guy who wrote Firesheep must be a world-class hacker, right?

Well, no. The work to package this up in a point-and-click way that is (sort of) accessible to power users is laudable, but what Firesheep actually does is far from magical. It’s more of an art project and PR stunt than an actual hack of any kind. Still, I was oddly excited to see Firesheep get so much PR, because it highlights a fundamental issue with the architecture of the web.

The web is kind of a primitive medium. The only way websites know who you are is through tiny, uniquely identifiying strings your browser sends to the webserver on each and every click:

GET / HTTP/1.1
Host: diy.stackexchange.com
Connection: keep-alive
User-Agent: Chrome/7.0.517.44
Accept-Language: en-US,en;q=0.8
Cookie: diyuser=t=ZlQOG4kege&s=8VO9gjG7tU12s
If-Modified-Since: Tue, 09 Nov 2010 04:41:12 GMT

These are the typical sort of HTTP headers your browser sends to a website on every click. See that little cookie in bright red? To a website, that’s your fingerprint, DNA, and social security number all rolled into one. Some part of the cookie contains a unique user ID that tells the website you are you.

And guess what? That cookie is always broadcast in plain text every single time you click a link on any website. Right out in the open where anyone — well, technically, anyone who happens to be on the same network as you and is in a position to view your network packets — can just grab it out of the ether and immediately impersonate you on any website you are a member of.

Now that you know how cookies work (and I’m not saying it’s rocket surgery or anything), you also know that what Firesheep does is relatively straightforward:

1. Listen to all HTTP traffic.
2. Wait for HTTP headers from a known website.
3. Isolate the part of the cookie header that identifies the user.
4. Launch a new browser session with that cookie. Bam! As far as the target webserver is concerned, you are that user!

All Firesheep has to do, really, is listen. That’s pretty much all there is to this “hack”. Scary, right? Well, then you should be positively quaking in your boots, because this is the way the entire internet has worked since 1994, when cookies were invented.

So why wasn’t this a problem in, say, 2003? Three reasons:

1. Commodity public wireless internet connections were not exactly common until a few years ago.
2. Average people have moved beyond mostly anonymous browsing and transferred significant parts of their identity online (aka the Facebook effect).
3. The tools required to listen in on a wireless network are slightly … less primitive now.

Firesheep came along at the exact inflection point of these three trends. And mind you, it is still not a sure thing — Firesheep requires a particular set of wireless network chipsets that support promiscuous mode in the lower level WinPcap library that Firesheep relies on. But we can bet that the floodgates have been opened, and future tools similar to this one will become increasingly a one-click affair.

The other reason this wasn’t a problem in 2003 is because any website that truly needed security switched to encrypted HTTP — aka Secure HTTP — long ago. HTTPS was invented in 1994, at the same time as the browser cookie. This was not a coincidence. The creators of the cookie knew from day one they needed a way to protect them from prying eyes. Even way, way back in the dark, primitive ages of 2003, any banking website or identity website worth a damn wouldn’t even consider using plain vanilla HTTP. They’d be laughed off the internet!

The outpouring of concern over Firesheep is justified, because, well, the web’s cookie jar has always been kind of broken — and we ought to do something about it. But what?

Yes, you can naively argue that every website should encrypt all their traffic all the time, but to me that’s a “boil the sea” solution. I’d rather see a better, more secure identity protocol than ye olde HTTP cookies. I don’t actually care if anyone sees the rest of my public activity on Stack Overflow; it’s hardly a secret. But gee, I sure do care if they somehow sniff out my cookie and start running around doing stuff as me! Encrypting everything just to protect that one lousy cookie header seems like a whole lot of overkill to me.

I’m not holding my breath for that to happen any time soon, though. So here’s what you can do to protect yourself, right now, today:

1. We should be very careful how we browse on unencrypted wireless networks. This is the great gift of Firesheep to all of us. If nothing else, we should be thanking the author for this simple, stark warning. It’s an unavoidable fact of life: if you must go wireless, seek out encrypted wireless networks. If you have no other choices except unencrypted wireless networks, browse anonymously — quite possible if all you plan to do is casually surf the web and read a few articles — and only log in to websites that support https. Anything else risks identity theft.
2. Get in the habit of accessing your web mail through HTTPS. Email is the de-facto skeleton key to your online identity. When your email is compromised, all is lost. If your webmail provider does not support secure http, they are idiots. Drop them like a hot potato and immediately switch to one that does. Heck, the smart webmail providers already switched to https by default!
3. Lobby the websites you use to offer HTTPS browsing. I think we’re clearly past the point where only banks and finance sites should be expected to use secure HTTP. As more people shift more of their identities online, it makes sense to protect those identities by moving HTTPS from the domain of a massive bank vault door to just plain locking the door. SSL isn’t as expensive as it used to be, in every dimension of the phrase, so this is not an unreasonable thing to ask your favorite website for.

This is very broad advice, and there are a whole host of technical caveats to the above. But it’s a starting point toward evangelizing the risks and responsible use of open wireless networks. Firesheep may indeed have broken the web’s cookie jar. But it was kind of an old, beat up, cracked cookie jar in the first place. I hope the powers that be will use Firesheep as incentive to build a better online identity solution than creaky old HTTP cookies.

Go to Source

The minute I switch back to my regular keyboard, I go from being Usain Bolt to the Flash.

Touchscreens are great for passively browsing, as Scott Adams noted:

Another interesting phenomenon of the iPhone and iPad era is that we are being transformed from producers of content into consumers. With my BlackBerry, I probably created as much data as I consumed. It was easy to thumb-type long explanations, directions, and even jokes and observations. With my iPhone, I try to avoid creating any message that are over one sentence long. But I use the iPhone browser to consume information a hundred times more than I did with the BlackBerry. I wonder if this will change people over time, in some subtle way that isn’t predictable. What happens when people become trained to think of information and entertainment as something they receive and not something they create?

Because we run an entire network of websites devoted to learning by typing words on a page, it’s difficult for me to get past this.

But I’m not here to decry the evils of touchscreen computing. It has its place in the pantheon of computing. I’m here to sing the praises of the humble keyboard. The device that, when combined with the internet, turns every human being into a highly efficient global printing press.

My love affair with the keyboard goes way back:

• Keyboarding
• We Are Typists First, Programmers Second
• Have Keyboard, Will Program

Maybe I’m biased. As I recently remarked on programmers.stackexchange.com, I can’t take slow typists seriously as programmers. When was the last time you saw a hunt-and-peck pianist?

I’ve been monogamous with the Microsoft Natural Keyboard 4000 for a long time. But in this supposedly happy marriage, I was accidentally neglecting one of the most crucial aspects of the keyboard experience.

The vast majority of keyboards included with white box systems or sold at office supply stores are rubber dome or membrane keyboards.

They are inexpensive, mass produced, relatively low quality devices that are inconsistent and degrade the user experience. Most users don’t know this, or simply don’t care. The appeal of cheap rubber dome or membrane keyboards is that they’re usually available in a variety of styles, are included “free” with a new system, and they may sport additional features like media controls or wireless connectivity. But these cheap keyboards typically don’t provide users with any tactile feedback, the keys feel mushy and may not all actuate at the same point, and the entire keyboard assemblies themselves tend to flex and move around when typed on. Not fun.

All this time, I’ve been typing on keyboards with least-common-denominator rubber dome innards. I was peripherally aware of higher quality mechanical keyboards, but I never appreciated them until I located this absolutely epic mechanical keyboard guide thread. It’s also the source of an entire forum of people at geekhack.org who are mechanical keyboard enthusiasts. These kinds of communities and obsessions, writ so large and with such obvious passion, fascinate me. They are the inspiration for what we are trying to do with Stack Overflow and the Stack Exchange network.

If you don’t have time to read that epic guide (but you should!), allow me to summarize:

1. Almost all computer and laptop keyboards today use cheap, low quality switches — rubber dome, membrane, scissor, or foam element.
2. Mechanical switches are considered superior in every way by keyboard enthusiasts.
3. Because the general public largely doesn’t care about keyboard feel or durability, and because mechanical switches are are more expensive, mechanical switch keyboards are quite rare these days.

Mechanical switches look, well, mechanical. They’re spiritually the same as those old-school arcade buttons we used to mash on in the 1980s. You push down on the key, and the switch physically actuates.

Yes, we are rapidly approaching the threshold of esoterica here. Mechanical keyboards were already becoming rare even before the internet, so I’d wager many people now reading this can’t possibly know the difference between a typical cheap membrane keyboard and a fancy mechanical model because they’ve never had the opportunity to try one!

We should rectify that.

If you want to dip your fingers into the world of mechanical switch keyboards, start by asking yourself a few questions:

• Are you willing to spend $70 to $300 for a keyboard?
• How noisy do you want your typing to be?
• Do you want a tactile “snap” when the key is depressed?
• How much force do you type with — do you have a light or heavy touch?
• How much key travel do you want?

Next, there are further subtleties to consider, like how the keys are printed:

• Pad Printed — the standard cheap stuff. Little more than stickers. Keycaps will wear off fast.
• Laser Etched — permanent, but leaves tiny surface scars on the keys due to the characters being literally burned into the keys. May also be a tiny bit blurry.
• Dye Sublimated — dye set into plastic; expensive but nearly optimal.
• Injection Molded — two keys in different colors are physically bonded together. Very expensive but considered as close to perfect as you can get. Notably, NeXT keyboards used this method.

And what about the shape of the keycaps? Cylindrical? Spherical? Flat? And if you’re an avid keyboard gamer, you might want to consider n-key rollover, too. I warned you this rabbit hole was deep.

Let’s start looking at a few likely candidates. The one you may already know is Das Keyboard.

Das is a good, reliable brand of mechanical keyboards. They have two primary models. Each is available in the “blank keycaps” versions if you are the sort of ninja typist who doesn’t need to look at the keyboard — you type by chanelling the Force.

• DAS Keyboard Model S Professional silent
• DAS Model S Professional

The “silent” mechanical switch distinction is an important one: mechanical switches can be loud. How loud? The DAS website actually sells honest-to-god earplugs as a keyboard accessory. I’m sure it’s slightly tongue in cheek. Maybe. But consider yourself warned, and choose the silent model if you aren’t a fan of the clickety-clack typing.

If you want the most old-school IBM-esque experience possible, and a true classic buckling spring keyboard, then Unicomp is your huckleberry. The common models are the Customizer 104/105 and SpaceSaver 104/105.

Next up is Elite Keyboards, but I can only recommend the (slightly expensive) Topre Realforce model due to the cheap pad keycap printing used on their other models.

Finally, Deck Keyboards — I remember writing about these guys years ago. They have a full sized keyboard now with a lot of attention to detail: The Deck Legend.

It is also the only keyboard in its class that is backlit, if that’s your bag.

Of course, none of these premium fancypants mechanical switch keyboards are really necessary. The most important aspect of writing isn’t the keyboard you use, but the simple act of getting out there and writing as much as you can. But if, like me, you accidentally fall in love with the keyboard and everything it represents — then I think you owe it to yourself to find out what a great keyboard is supposed to feel like.

Go to Source

Those days are long gone. A router is one of those salt-of-the-earth items now; anyone who pays for an internet connection needs a router, for:

1. NAT and basic hardware firewall protection from internet evildoers
2. A wired network hub to connect local desktop PCs
3. A wireless hub to connect laptops, phones, consoles, etcetera

Let me put it this way: my mom — and my wife’s mom — both own routers. If that isn’t the definition of mainstream, I don’t know what is.

Since my livelihood revolves around being on the internet, and because I’m a bit of a tweaker, I have a fancy-ish router. But it is of late 2007 vintage:

Although the DGL-4500 is a nice router, and it has served me well with no real complaints, the last major firmware update for it was a year and a half ago. There have been some desultory minor updates since then, but clearly the vendor has, shall we say, moved on to focusing on newer models.

The router is (literally!) the central component in my overall internet experience, and I was growing increasingly uncomfortable with the status quo. Frankly, the prospect of three year old hardware with year old firmware gives me the heebie-jeebies.

So, I asked the pros at Super User, even going so far as to set up a Recommend Me a Router chat room. (We disallow product recommendation questions as they become uselessly out of date so quickly, but this is a perfect topic for a chat room.) I got some fantastic advice from my fellow Super Users via chat, though much of it was of the far too sane “if it ain’t broke don’t fix it” variety. Well, that’s just not how I work. To be fair, the router market is not exactly a hotbed of excitement at the moment; it is both saturated and heavily commoditized, particularly now that the dust has settled from the whole 802.11 A/B/G/N debacle. There just isn’t much going on.

But in the process of doing my router research, I discovered something important, and maybe even revolutionary in its own quiet little way. The best router models all run open source firmware!

• DD-WRT
• Tomato

That’s right, the truly great routers are available in “awesome” edition. (There may be other open source router firmwares out there, but these are the two I saw most frequently.) I learned that these open source firmwares can turn a boring Clark Kent router into Superman. And they are always kept updated by the community, in perpetuity.

In my weaker moments, I toyed with the idea of building a silent mini x86 PC that could run a routing optimized distribution of Linux, but the reality is that current commodity routers have more than enough memory and embedded CPU power — not to mention the necessary wireless and gigabit ethernet hub bits already built in. Dedicating a whole x86 PC to routing is power inefficient, overly complex, and awkward.

Yes, today’s router marketplace is commoditized and standardized and boring — but there are still a few clear hardware standouts. I turned to the experts at SmallNetBuilder for their in-depth technical reviews, and found two consensus recommendations:

Buffalo Nfiniti Wireless-N High Power Router ($80)

NETGEAR WNDR3700 RangeMax Dual Band Wireless-N ($150)

Both of these models got glowing reviews from the networking experts at SmallNetBuilder, and both are 100% compatible with the all-important open source dd-wrt firmware. You can’t go wrong with either, but I chose the less expensive Buffalo Nfiniti router. Why?

1. It’s almost half the price, man!
2. The “high power” part is verifiably and benchmarkably true, and I have some wireless range problems at my home.
3. I do most of my heavy network lifting through wired gigabit ethernet, so I can’t think of any reason I’d need the higher theoretical wireless throughput of the Netgear model.
4. Although the Netgear has a 680 Mhz embedded CPU and 128mb RAM, the Buffalo’s 400 MHz embedded CPU and 64mb of RAM is not exactly chopped liver, either; it’s plenty for dd-wrt to work with. I’d almost go so far as to say the Netgear is a bit overkill… if you’re into that sort of thing.

I received my Buffalo Nfiniti and immediately installed dd-wrt on it, which was very simple and accomplished through the existing web UI on the router. (Buffalo has a history of shipping rebranded dd-wrt distributions in their routers, so the out-of-box firmware is a kissing cousin.)

After rebooting, I was in love. The (more) modern gigabit hardware, CPU, and chipset was noticably snappier everywhere, even just dinking around in the admin web pages. And dd-wrt scratches every geek itch I have — putting that newer hardware to great use. Just check out the detailed stats I can get, including that pesky wireless signal strength problem. The top number is the Xbox 360 outside, the bottom number is my iPhone from about 10 feet away.

Worried your router is running low on embedded CPU grunt, or that 64 megabytes of memory is insufficient? Never fear; dd-wrt has you covered. Just check out the detailed, real time memory and cpu load stats.

Trying to figure out how much WAN/LAN/Wireless bandwidth you’re using? How does a real time SVG graph, right from the router admin pages, grab you?

It’s just great all around. And I haven’t even covered the proverbial laundry list of features that dd-wrt offers above and beyond most stock firmware! Suffice it to say that this is one of those times when the “let’s support everything” penchant of open source projects works in our favor. Don’t worry, it’s all (mostly) disabled by default. Those features and tweaks can all safely be ignored; just know that they’re available to you when and if you need them.

This is boring old plain vanilla commodity router hardware, but when combined with an open source firmware, it is a massive improvement over my three year old, proprietary high(ish) end router. The magic router formula these days is a combination of commodity hardware and open-source firmware. I’m so enamored of this one-two punch combo, in fact, I might even say it represents the future. Not just of the everyday workhorse routers we all need to access the internet — but the future of all commodity hardware.

Routers; we all need ‘em, and they are crucial to our internet experience. Pick whichever router you like — as long as it’s compatible with one of the open source firmware packages! Thanks to a wide variety of mature commodity hardware choices, plus infinitely and perpetually updated open source router firmware, I’m happy to report that now everyone can have a great router.

Go to Source

Now think back through all the videos you’ve watched on YouTube. How many of them contained any original content?

It’s perhaps the ultimate case of cognitive dissonance: by YouTube’s own rules [which prohibit copyrighted content], YouTube cannot exist. And yet it does.

How do we reconcile YouTube’s official hard-line position on copyright with the reality that 90% of the content on their site is clearly copyrighted and clearly used without permission? It seems YouTube has an awfully convenient “don’t ask, don’t tell” policy– they make no effort to verify that the uploaded content is either original content or fair use. The copyrighted content stays up until the copyright owner complains. Then, and only then, is it removed.

Today’s lesson, then, is be careful what you ask for.

At the time, I just assumed that YouTube would never be able to resolve this problem through technology. The idea that you could somehow fingerprint every user-created uploaded video against every piece of copyrighted video ever created was so laughable to me that I wrote it off as impossible.

A few days ago I uploaded a small clip from the movie Better Off Dead to YouTube, in order to use it in the Go That Way, Really Fast blog entry. This is quintessential fair use: a tiny excerpt of the movie, presented in the context of a larger blog entry. So far, so good.

But then I uploaded a small clip from a different movie that I’m planning to use in another, future blog entry. Within an hour of uploading it, I received this email:

Dear {username},

Your video, YouTube vs. Fair Use, may have content that is owned or licensed by {company}.

No action is required on your part; however, if you are interested in learning how this affects your video, please visit the Content ID Matches section of your account for more information.

Sincerely,
- The YouTube Team

This 90 second clip is from a recent movie. Not a hugely popular movie, mind you, but a movie you’ve probably heard of. This email both fascinated and horrified me. How did they match a random, weirdly cropped (thanks, Windows Movie Maker) clip from the middle of a non-blockbuster movie within an hour of me uploading it? This had to be some kind of automated process that checks uploaded user content against every piece of copyrighted content ever created (or the top n subset thereof), exactly the kind that I thought was impossible.

Uh oh.

I began to do some research. I quickly found Fun with YouTube’s Audio Content ID System, which doesn’t cover video, but it’s definitely related:

I was caught by surprise one day when I received an automated email from YouTube informing me that my video had a music rights issue and it was removed from the site. I didn’t really care.

Then a car commercial parody I made (arguably one of my better videos) was taken down because I used an unlicensed song. That pissed me off. I couldn’t easily go back and re-edit the video to remove the song, as the source media had long since been archived in a shoebox somewhere. And I couldn’t simply re-upload the video, as it got identified and taken down every time. I needed to find a way to outsmart the fingerprinter. I was angry and I had a lot of free time. Not a good combination.

I racked my brain trying to think of every possible audio manipulation that might get by the fingerprinter. I came up with an almost-scientific method for testing each modification, and I got to work.

Further research led me to this brief TED talk, How YouTube Thinks About Copyright.

We compare each upload against all the reference files in our database. This heat map is going to show you how the brain of this system works.

Here we can see the reference file being compared to the user generated content. The system compares every moment of one to the other to see if there’s a match. This means we can identify a match even if the copy uses just a portion of the original file, plays it in slow motion, and has degraded audio or video.

The scale and speed of this system is truly breathtaking — we’re not just talking about a few videos, we’re talking about over 100 years of video every day between new uploads and the legacy scans we regularly do across all of the content on the site. And when we compare those 100 years of video, we’re comparing it against millions of reference files in our database. It’d be like 36,000 people staring at 36,000 monitors each and every day without as much as a coffee break.

I have to admit that I’m astounded by the scope, scale, and sheer effectiveness of YouTube’s new copyright detection system that I thought was impossible! Seriously, watch the TED talk. It’s not long. The more I researched YouTube’s video identification tool, the more I realized that resistance is futile. It’s so good that the only way to defeat it is by degrading your audio and video so much that you have effectively ruined it. And when it comes to copyright violations, if you can achieve mutually assured destruction, then you have won. Absolutely and unconditionally.

This is an outcome so incredible I am still having trouble believing it. But I have the automatically blocked uploads to prove it.

Now, I am in no way proposing that copyright is something we should be trying to defeat or work around. I suppose I was just used to the laissez faire status quo on YouTube, and the idea of a video copyright detection system this effective was completely beyond the pale. My hat is off to the engineers at Google who came up with this system. They aren’t the bad guys here; they offer some rather sane alternatives when copyright matches are found:

If Content ID identifies a match between a user upload and material in the reference library, it applies the usage policy designated by the content owner. The usage policy tells the system what to do with the video. Matches can be to only the audio portion of an upload, the video portion only, or both.

There are three usage policies — Block, Track or Monetize. If a rights owner specifies a Block policy, the video will not be viewable on YouTube. If the rights owner specifies a Track policy, the video will continue to be made available on YouTube and the rights owner will receive information about the video, such as how many views it receives. For a Monetize policy, the video will continue to be available on YouTube and ads will appear in conjunction with the video. The policies can be region-specific, so a content owner can allow a particular piece of material in one country and block the material in another.

The particular content provider whose copyright I matched chose the draconian block policy. That’s certainly not Google’s fault, but I guess you could say I’m Feeling Unlucky.

Although the 90 second clip I uploaded is clearly copyrighted content — I would never dispute that — my intent is not to facilitate illegal use, but to “quote” the movie scene as part of a larger blog entry. YouTube does provide recourse for uploaders; they make it easy to file a dispute once the content is flagged as copyrighted. So I dutifully filled out the dispute form, indicating that I felt I had a reasonable claim of fair use.

Unfortunately, my fair use claim was denied without explanation by the copyright holder.

Let’s consider the four guidelines for fair use I outlined in my original 2007 blog entry:

1. Is the use transformative?
2. Is the source material intended for the public good?
3. How much was taken?
4. What’s the market effect?

While we’re clear on 3 and 4, items 1 and 2 are hazy in a mashup. This would definitely be transformative, and I like to think that I’m writing for the erudition of myself and others, not merely to entertain people. I uploaded with the intent of the video being viewed through a blog entry, with YouTube as the content host only. But it was still 90 seconds of the movie viewable on YouTube by anyone, context free.

So I’m torn.

On one hand, this is an insanely impressive technological coup. The idea that YouTube can (with the assistance of the copyright holders) really validate every minute of uploaded video against every minute of every major copyrighted work is unfathomable to me. When YouTube promised to do this to placate copyright owners, I was sure they were delaying for time. But much to my fair-use-loving dismay, they’ve actually gone and built the damn thing — and it works.

Just, maybe, it works a little too well. I’m still looking for video sharing services that offer some kind of fair use protection.

Go to Source

1. A solid state hard drive is easily the best and most obvious performance upgrade you can make on any computer for a given amount of money. Unless your computer is absolute crap to start with.
2. The practical minimum solid state hard drive size, 128 GB, has modestly declined in price — from about $350 to about $250.

(yes, you can get by with 64 GB, but at least with my Windows installs I find that I have to think about disk space with 64 GB, whereas with 128 GB I don’t have to worry — ever. Don’t make me think, man!)

The rest of the components inside your PC are downright boring in comparison. CPUs? All stupid fast at any price, with more cores and gigahertz than you’ll ever need unless you’re one of those freaks who does nothing but raytracing and video transcoding all day long. Memory? Dirt cheap, and average users won’t need more than 2 gigabytes of the stuff in practical use, which at the current going rate for DDR3 is less than 50 bucks.

Thanks to the neverending march of Moore’s Law, PCs are becoming speedy at any price these days. It’s difficult for me to muster any enthusiasm for the latest Intel CPU updates when I spend almost zero real world time waiting for the CPU to do anything on my computer. I guess it’s true: absolute power corrupts absolutely.

But hard drives, now, there’s where you can pay a bit more and see a groundbreaking, generational leap in performance worthy of that investment — as long as you skip over the old and busted spinning rust hard drives, and choose a newfangled solid state hard drive.

The current king of the hill seems to be the Crucial RealSSD C300.

Pretty sexy, right? Oh, who am I kidding, it’s a boring slab of aluminum and silicon. But like all truly sexy things, what turns me on is the part I can’t see — the sexy, sexy, sexy performance inside this baby.

See those bars dragging down the bottom of this graph? All spinning rust. Heck, even the SSD I recommended last year is only middle of the pack here. AnandTech concurs — the Crucial RealSSD C300 is top dog, at least for now.

(Be careful, though, that your operating system supports the SSD TRIM command, otherwise you’ll suffer severe performance degradation over time with almost any SSD. Operating systems earlier than Windows 7 and the latest, greatest Linux kernel should beware — and, shockingly, OSX still doesn’t support TRIM!)

Where it gets trickier, though, is when you need more than 128 GB of storage, or when you are limited to one 2.5″ hard drive — like in a laptop. In that case, ideally you’d like something that has the speed of a solid state hard drive, but the capacity (and economical price per gigabyte) of a traditional magnetic platter hard drive. You might say … a hybrid hard drive, the kind I was dreaming about back in 2006 …

After all this analysis, it’s clear to me that traditional hard drives and flash memory are quite complimentary; they’re strong in different areas. But flash drives are the future. They will definitely replace hard drives in almost all low end and low power devices– and future high performance hard drives will need to have a substantial chunk of flash memory on board to stay competitive.

I had always been disappointed that hybrid hard drives, drives that combine both flash memory and traditional magnetic platters, never came to fruition. It was either traditional or SSD and nothing in between. It seemed like such an obvious “best of both worlds” scenario to me. But I recently discovered that decent hybrid drives do finally exist — though in a small and mostly unheralded way.

Seagate’s Momentus XT takes a totally respectable 2.5″, 7200 RPM drive with a 32 megabyte buffer and combines it with 4 gigabytes of flash memory. The result is the exactly what I had always hoped:

Seagate’s Momentus XT should become the standard hard drive in any notebook shipped. The biggest problem I have with using any brand new machine, regardless of how fast it is, is that it never feels fast because it usually has a HDD and not an SSD. While the Momentus XT isn’t quite as fast as an SSD, it’s a significant improvement over the mechanical drives found in notebooks today.

In many cases the Momentus XT performs like a VelociRaptor, but in a lower power, quieter package. The impact of adding just a small amount of SLC NAND is tremendous. The potential for hybrid drives continues to be huge; what Seagate has shown here is that with a minimal amount of NAND you can achieve some tremendous performance gains.

And the best part? 500 gigabytes of near-SSD performance for $130! Or, if that’s too spendy, how about 320 gigabytes of near-SSD performance for $99?

I’ve ordered a few of these drives to upgrade my laptops and home theater PC. Sure, I’ll invest in a SSD for my beastly desktop, but I can’t justify $300 to put a SSD in a laptop I spent all of $800 on, or a home theater PC that set me back a mere $500. But a hundred bucks for near-SSD performance and decent capacity? Sign me up. And hard drive vendors: although I love SSDs to death, please keep these improved hybrid drives coming, too, please!

Go to Source

More specifically, Curtis Armstrong as Charles De Mar from the 1985 absurdist teen comedy classic, Better Off Dead. When asked for advice on how to ski down a particularly treacherous mountain, he replied:

Go that way, really fast. If something gets in your way … turn.

(I recommend watching the video clip, or, better yet, the entire movie. It’s brilliant in ways I can’t possibly explain here.)

In the five months since we announced our funding, we have …

• Built an international team
• Created an entirely new open, democratic process for creating Q&A sites at Area 51
• Launched ~24 new community-driven Stack Exchange network sites
• Implemented per-site meta discussion and per-site real time chat
• Rolled out new versions of Careers and Jobs
• Built and open-sourced a tool for exploring and sharing all our creative commons data in the Stack Exchange Data Explorer
• Finalized V1 of the Stack Exchange API, for building your own apps against our Q&A platform

… and honestly, I’m a little worried we’re still not going fast enough.

There are any number of Stack Overflow engine clones out there already, and I say more power to ‘em. I’m proud to have something worth copying. If we do nothing else except help lead the world away from the ancient, creaky, horribly broken bulletin board model of phpBB and vBulletin — attempting to get information out of those things is like panning for gold in a neverending river of sewage — then that is more than I could have ever hoped for.

It is our stated goal as a company to live in harmony with the web, by only doing things that we believe make the internet better, at least in some small way. No, seriously. It’s in writing and everything, I swear! We’re not here to subvert or own anyone or anything. We just love community, and we love getting great answers to our questions. So if something gets in our way while doing that, well, we’re not gonna fight you. We’ll just turn. And keep going forward, really fast. Which is why those clones better move quick if they want to keep up with us.

While I like to think that having Charles De Mar as a business advisor is unique to our company, the idea that speed is important is hardly original to us. For example, certain Google projects also appear to understand Boyd’s Law of Iteration.

Boyd decided that the primary determinant to winning dogfights was not observing, orienting, planning, or acting better. The primary determinant to winning dogfights was observing, orienting, planning, and acting faster. In other words, how quickly one could iterate. Speed of iteration, Boyd suggested, beats quality of iteration.

Speed of iteration — the Google Chrome project has it.

Chrome was a completely respectable browser in V1 and V2. The entire project has moved forward so fast that it now is, at least in my humble opinion, the best browser on the planet. Google went from nothing, no web browser at all, to best-of-breed in under two years. Meanwhile, Internet Explorer took longer than the entire development period of Chrome to go from version 7 to version 8. And by the time Internet Explorer 9 ships — even though it’s actually looking like Microsoft’s best, most competent technical upgrade of the browser yet — it will be completely outclassed at launch by both Firefox and Chrome.

The Google Android project is another example. Android doesn’t have to be better than the iPhone (and it most definitely isn’t; it’s been mediocre at best until recent versions). They just need to be faster at improving. Google is pushing out Froyos and Gingerbreads and Honeycombs with incredible, breakneck speed. Yes, Apple has indisputably better taste — and an impeccably controlled experience. But at their current rate of progress, they’ll be playing second or third fiddle to Google in the mobile space inside a few years. It’s inevitable.

So, until further notice, we’ll be following the same strategy as the Android and Chrome teams. We’re going to go that way, really fast. And if something gets in our way, we’ll turn.

Go to Source

A question that comes up again and again in web development companies is:

“Should the developers have access to the production environment, and if they do, to what extent?”

My view on this is that as a whole they should have limited access to production. A little
disclaimer before I attempt to justify this view is that this standpoint is in no way based on the perceived quality or attitude of the developers — so please don’t take it this way.

This is a tricky one for me to answer, because, well, I’m a developer. More specifically, I’m one of the developers Kyle is referring to. How do I know that? Because Kyle works for our company, Stack Overflow Internet Services Incorporated©®™. And Kyle is a great system administrator. How do I know that? Two reasons:

1. He’s one of the top Server Fault users.
2. He had the audacity to write about this issue on the Server Fault blog.

From my perspective, the whole point of the company is to talk about what we’re doing. Getting things done is important, of course, but we have to stop occasionally to write up what we’re doing, how we’re doing it, and why we’re even doing it in the first place — including all our doubts and misgivings and concerns. If we don’t, we’re cheating ourselves, and you guys, out of something much deeper. Yes, writing about what we’re doing and explaining it to the community helps us focus. It lets our peers give us feedback. But most importantly of all, it lets anyone have the opportunity to learn from our many, many mistakes … and who knows, perhaps even the occasional success.

That’s basically the entire philosophy behind our Stack Exchange Q&A network, too. Let’s all talk about this stuff in public, so that we can teach each other how to get better at whatever the heck it is we love to do.

(Sometimes I get the feeling this idea makes my co-founder nervous, which I continually struggle to understand. If we don’t walk the walk, why are we even doing this? But I digress.)

The saga of System Administrators versus Programmers is not a new one; I don’t think I’ve ever worked at any company where these two factions weren’t continually battling with each other in some form. It’s truly an epic struggle, but to understand it, you have to appreciate that both System Administrators and Programmers have different, and perhaps complementary, supernatural powers.

Programmers are like vampires. They’re frequently up all night, paler than death itself, and generally afraid of being exposed to daylight. Oh yes, and they tend think of themselves (or at least their code) as immortal.

System Administrators are like werewolves. They may look outwardly ordinary, but are incredibly strong, mostly invulnerable to stuff that would kill regular people — and prone to strange transformations during a moon “outage”.

Let me be very clear that just as Kyle respects programmers, I have a deep respect for system administrators:

Although there is certainly some crossover, we believe that the programming community and the IT/sysadmin community are different beasts. Just because you’re a hotshot programmer doesn’t mean you have mastered networking and server configuration. And I’ve met a few sysadmins who could script circles around my code. That’s why Server Fault gets its own domain, user profiles, and reputation system.

Different “beasts” indeed.

Anyway, if you’re looking for a one size fits all answer to the question of how much access programmers should have to production environments, I’m sorry, I can’t give you one. Every company is different, every team is different. I know, it’s a sucky answer, but it depends.

However, as anyone who has watched the latest season of True Blood (or, God help us all, the Twilight Eclipse movie) can attest, there are ways for vampires and werewolves to work together. In a healthy team, everyone feels their abilities are being used and not squandered.

On our team, we’re all fair-to-middling sysadmins. But there are a million things to do, and having a professional sysadmin means we can focus on the programming while the networking, hardware, and operational stuff gets a whole lot more TLC and far better (read: non-hacky) processes put in place. We’re happy to refocus our efforts on what we’re expert at, and let Kyle put his skills to work in areas that he’s expert at. Now, that said, we don’t want to cede full access to the production servers — but there’s a happy middle ground where our access becomes infrequent and minor over time, except in the hopefully rare event of an all hands on deck emergency.

The art of managing vampires and werewolves, I think, is to ensure that they spend their time not fighting amongst themselves, but instead, using those supernatural powers together to achieve a common goal they could not otherwise. In my experience, when programmers and system administrators fight, it’s because they’re bored. You haven’t given them a sufficiently daunting task, one that requires the full combined use of their unique skills to achieve.

Remember, it’s not vampires versus werewolves. It’s vampires and werewolves.

Go to Source

But on those unfortunate occasions when I am compelled — for reasons entirely beyond my control — to leave the house, I do so fully armed with my crucial utility belt items. Yes, you heard me, I transform from the geeky Bruce Wayne to the gosh-darned Batman!

At least, that’s how I like to think of it.

I’ve been talking about this every-day carry stuff for quite a while now. The 2010 edition of my personal utility belt is mostly subtle tweaks, but I daresay it’s the best one yet.

The art of every-day carry must go on. What you see here is the contents of my pocket:

1. Patriot 32 GB USB flash drive

   Now you can have a whole freakin’ hard drive worth of files in your pocket. Just in case, you know, you have an emergency need to upload a virus to an alien mothership, or something. Beware the many cheap, slow USB flash drives out there; this one is a real gem. It’s inexpensive, and per my measurements, about as fast as they get. This is important because the larger the flash drive, the more important speed becomes. Hard to believe I’ve gone from carrying a 512 megabyte flash drive in 2005 to a 32 gigabyte flashdrive in 2010.

2. Leatherman Squirt P4

   Ounce for ounce, nothing beats the utility of the Leatherman Squirt. This time I opted for the plier (P) version instead of the scissors (S), and after seeing how much more generally useful the pliers are, I am now a little ashamed to admit I ever carried the wussy scissors version. Pliers all the way, baby. And yes, that is a Pulp Fiction joke you see on it.

3. Fenix mini AAA LED flashlight model LD01R2

   Since I’ve been carrying them in 2005, the average LED flashlight has gone from bright, to very bright, to amazingly bright, to ridiculously blinding laser-like bright. It’s scary how bright these fancy milled aluminum AAA LED flashlights get now. What I like about this one is that it lets you trade off stupid-brightness for something practical, like greater runtime: you can twist the top to switch levels: 9 Lumens for 11 hours, 28 Lumens for 3.5 hours, or 85 Lumens for 1 hour.

4. Small Nite Ize s-biner

   These little nite-ize carabiners are awesome for quick attachment and detachment of your EDC items, but I’ll warn you: resist the urge to put everything you carry on a carabiner, because if you do, the weight and “jangliness” goes up a lot — and this way lies madness. Consider how many items you actually remove from your keychain regularly. For me, the only item I frequently removed to work with was the Squirt, so that’s the only one I put on a carabiner.

Rest assured, everything here is carefully selected with the appropriate levels of monomaniacal attention to detail. For this weight and size, I don’t think you can do better. (And don’t think I’ve forgotten about optimizing my wallet, either. Oh no. Quite the contrary.)

However, I have to add a special category this year for the other must-have EDC utility belt item: the smartphone. What self-respecting superhero would leave the house these days without their smartphone? I’m not religious about it, but I use and rather like the iPhone 4, and I’m continually amazed how many things it does that I used to carry separate items for:

• cell phone (obviously)
• “Nintendo DSwhatever” for portable gaming
• GPS
• point and shoot digital camera
• near-desktop quality mobile web browser and email client
• mp3 player with speakers
• audio and hi-def video recorder
• DVD player
• ebook reader
• watch, alarm
• emergency flashlight (via front facing LED flash control)
• scanner
• level and ruler

Smartphones really are the ultimate gadget. The list of functions is already enormous, and I’m sure I’m leaving out a few other things that you can do with a modern smartphone.

In a pinch, I could conceivably drop the AAA LED flashlight and the USB flash drive from my EDC kit and substitute the smartphone. Not exactly, mind you, but it’s getting closer every year. At this rate, Apple could introduce a flip-out blade on the iPhone 7 and reduce my entire EDC kit to one item.

Anyway, that’s what’s on my utility belt in 2010. What’s on yours?

Go to Source

If you aren’t familiar with this classic film, the premise is simple: Bill Murray, somehow, gets stuck reliving the same day over and over.

It’s been at least 5 years since I’ve seen Groundhog Day. I don’t know if it’s my advanced age, or what, but it really struck me on this particular viewing: this is no comedy. There’s a veneer of broad comedy, yes, but lurking just under that veneer is a deep, dark existential conundrum.

It might be amusing to relive the same day a few times, maybe even a few dozen times. But an entire year of the same day — an entire decade of the same day — everything happening in precisely, exactly the same way? My back of the envelope calculation easily ran to a decade. But I was wrong. The director, Harold Ramis thinks it was actually 30 or 40 years.

I think the 10-year estimate is too short. It takes at least 10 years to get good at anything, and alloting for the down time and misguided years [Phil] spent, it had to be more like 30 or 40 years [spent reliving the same day].

We only see bits and pieces of the full experience in the movie, but this time my mind began filling in the gaps. Repeating the same day for decades plays to our secret collective fear that our lives are irrelevant and ultimately pointless. None of our actions — even suicide, in endless grisly permutations — ever change anything. What’s the point? Why bother? How many of us are trapped in here, and how can we escape?

This is some dark, scary stuff when you really think about it.

You want a prediction about the weather, you’re asking the wrong Phil.

I’ll give you a winter prediction.
It’s gonna be cold,
it’s gonna be gray,
and it’s gonna last you for the rest of your life.

Comedy, my ass. I wanted to cry.

But there is a way out: redemption through repetition. If you have to watch Groundhog Day a few times to appreciate it, you’re not alone. Indeed, that seems to be the whole point. Just ask Roger Ebert:

“Groundhog Day” is a film that finds its note and purpose so precisely that its genius may not be immediately noticeable. It unfolds so inevitably, is so entertaining, so apparently effortless, that you have to stand back and slap yourself before you see how good it really is.

Certainly I underrated it in my original review; I enjoyed it so easily that I was seduced into cheerful moderation. But there are a few films, and this is one of them, that burrow into our memories and become reference points. When you find yourself needing the phrase This is like “Groundhog Day” to explain how you feel, a movie has accomplished something.

There’s something delightfully Ouroboros about the epiphanies and layered revelations in repeated viewings of a movie that is itself about (nearly) endless repetition.

Which, naturally, brings me to A/B testing. That’s what Phil spends most of those thirty years doing. He spends it pursuing a woman, technically, but it’s how he does it that is interesting:

Rita: This whole day has just been one long setup.

Phil: It hasn’t.

Rita: And I hate fudge!

Phil: [making a mental list] No white chocolate. No fudge.

Rita: What are you doing? Are you making some kind of list? Did you call my friends and ask what I like and what I don’t like? Is this what love is for you?

Phil: This is real. This is love.

Rita: Stop saying that! You must be crazy.

Phil doesn’t just go on one date with Rita, he goes on thousands of dates. During each date, he makes note of what she likes and responds to, and drops everything she doesn’t. At the end he arrives at — quite literally — the perfect date. Everything that happens is the most ideal, most desirable version of all possible outcomes on that date on that particular day. Such are the luxuries afforded to a man repeating the same day forever.

This is the purest form of A/B testing imaginable. Given two choices, pick the one that “wins”, and keep repeating this ad infinitum until you arrive at the ultimate, most scientifically desirable choice. Your marketing weasels would probably collapse in an ecstatic, religious fervor if they could achieve anything even remotely close to the level of perfect A/B testing depicted in Groundhog Day.

But at the end of this perfect date, something impossible happens: Rita rejects Phil.

Phil wasn’t making these choices because he honestly believed in them. He was making these choices because he wanted a specific outcome — winning over Rita — and the experimental data told him which path he should take. Although the date was technically perfect, it didn’t ring true to Rita, and that made all the difference.

That’s the problem with A/B testing. It’s empty. It has no feeling, no empathy, and at worst, it’s dishonest. As my friend Nathan Bowers said:

A/B testing is like sandpaper. You can use it to smooth out details, but you can’t actually create anything with it.

The next time you reach for A/B testing tools, remember what happened to Phil. You can achieve a shallow local maximum with A/B testing — but you’ll never win hearts and minds. If you, or anyone on your team, is still having trouble figuring that out, well, the solution is simple.

Just watch Groundhog Day again.

Go to Source

Using a mouse or keyboard to control a computer? Don’t be silly. In the future, clearly there’s only one way computers will be controlled: by speaking to them.

There’s only one teeny-tiny problem with this magical future world of computers we control with our voices.

It doesn’t work.

Despite ridiculous, order of magnitude increases in computing power over the last decade, we can’t figure out how to get speech recognition accuracy above 80% — when the baseline human voice transcription accuracy rate is anywhere from 96% to 98%!

In 2001 recognition accuracy topped out at 80%, far short of HAL-like levels of comprehension. Adding data or computing power made no difference. Researchers at Carnegie Mellon University checked again in 2006 and found the situation unchanged. With human discrimination as high as 98%, the unclosed gap left little basis for conversation. But sticking to a few topics, like numbers, helped. Saying “one” into the phone works about as well as pressing a button, approaching 100% accuracy. But loosen the vocabulary constraint and recognition begins to drift, turning to vertigo in the wide-open vastness of linguistic space.

As Robert Fortner explained in Rest in Peas: The Unrecognized Death of Speech Recognition, after all these years, we’re desperately far away from any sort of universal speech recognition that’s useful or practical.

Now, we do have to clarify that we’re talking about universal recognition: saying anything to a computer, and having it reliably convert that into a valid, accurate text representation. When you constrain the voice input to a more limited vocabulary — say, just numbers, or only the names that happen to be in your telephone’s address book — it’s not unreasonable to expect a high level of accuracy. I tend to think of this as “voice control” rather than “voice recognition”.

Still, I think we’re avoiding the real question: is voice control, even hypothetically perfect voice control, more effective than the lower tech alternatives? In my experience, speech is one of the least effective, inefficient forms of communicating with other human beings. By that, I mean …

• typical spoken communication tends to be off-the-cuff and ad-hoc. Unless you’re extremely disciplined, on average you will be unclear, rambling, and excessively verbose.
• people tend to hear about half of what you say at any given time. If you’re lucky.
• spoken communication puts a highly disproportionate burden on the listener. Compare the time it takes to process a voicemail versus the time it takes to read an email.

I am by no means against talking with my fellow human beings. I have a very deep respect for those rare few who are great communicators in the challenging medium of conversational speech. Though we’ve all been trained literally from birth how to use our voices to communicate, voice communication remains filled with pitfalls and misunderstandings. Even in the best of conditions.

So why in the world — outside of a disability — would I want to extend the creaky, rickety old bridge of voice communication to controlling my computer? Isn’t there a better way?

Robert’s post contains some examples in the comments from voice control enthusiasts:

in addition to extremely accurate voice dictation, there are those really cool commands, like being able to say something like “search Google for Balloon Boy” or something like that and having it automatically open up your browser and enter the search term — something like this is accomplished many times faster than a human could do it. Or, being able to total up a column of numbers in Microsoft Excel by saying simply “total this column” and seeing the results in a blink of an eye, literally.

That’s funny, because I just fired up the Google app on my iPhone, said “balloon boy” into it, and got .. a search for “blue boy”. I am not making this up. As for the Excel example, total which column? Let’s assume you’ve dealt with the tricky problem of selecting what column you’re talking about with only your voice. (I’m sorry, was it D5? B5?) Wouldn’t it be many times faster to click the toolbar icon with your mouse, or press the keyboard command equivalent, to sum the column — rather than methodically and tediously saying the words “sum this column” out loud?

I’m also trying to imagine a room full of people controlling their computers or phones using their voices. It’s difficult enough to get work done in today’s chatty work environments without the added burden of a floor full of people saying “zoom … enhance” to their computers all day long. Wouldn’t we all end up hoarse and deaf?

Let’s look at another practical example — YouTube’s automatic speech recognition feature. I clicked through to the first UC Berkeley video with this feature, clicked the CC (closed caption) icon, and immediately got .. this.

“Light exerts force on matter”. But according to Google’s automatic speech recognition, it’s “like the search for some matter”. Unsurprisingly, it does not get better from there. You’d be way more confused than educated if you had to learn this lecture from the automatic transcription.

Back when Joel Spolsky and I had a podcast together, a helpful listener suggested using speech recognition to get a basic podcast transcript going. Everything I knew about voice recognition told me this wouldn’t help, but harm. What’s worse: transcribing everything by hand, from scratch — or correcting every third or fourth word in an auto-generated machine transcript? Maybe it’s just me, but the friction of the huge error rate inherent in the machine transcript seems far more intimidating than a blank slate human transcription. The humans may not be particularly efficient, but they all add value along the way — collective human judgment can editorially improve the transcript, by removing all the duplication, repetition, and “ums” of a literal, by-the-book transcription.

In 2004, Mike Bliss composed a poem about voice recognition. He then read it to voice recognition software on his PC, and rewrote it as recognized.

The real punchline here is that Mike re-ran the experiment in 2008, and after 5 minutes of voice training, the voice recognition got all but 2 words of the original poem correct!

I suspect that’s still not good enough in the face of the existing simpler alternatives. Remember handwriting recognition? It was all the rage in the era of the Apple Newton.

It wasn’t as bad as Doonesbury made it out to be. I learned Palm’s Graffiti handwriting recognition language and got fairly proficient with it. More than ten years later, you’d expect to see massively improved handwriting recognition of some sort in today’s iPads and iPhones and iOthers, right? Well, maybe, if by “massively improved” you mean “nonexistent”.

While it still surely has its niche uses, I personally don’t miss handwriting recognition. Not even a little. And I can’t help wondering if voice recognition will go the same way.

Go to Source

Well, I have a confession to make: my heart was never in Stack Exchange. It was a parallel effort in a parallel universe only tangentially related to my own. There’s a whole host of reasons why, but if I had to summarize it in a sentence, I’d say that money is poisonous to communities. That $129/month doesn’t sound like much — and it isn’t — but the commercial nature of the enterprise permeated and distorted everything from the get-go.

(fortunately, the model is changing with Stack Exchange 2.0, but that’s a topic for another blog post.)

Yes, Stack Overflow Internet Services Incorporated©®™ is technically a business, even a venture capital backed business now — but I didn’t co-found it because I wanted to make money. I co-founded it because I wanted to build something cool that made the internet better. Yes, selfishly for myself, of course, but also in conjunction with all of my fellow programmers, because I know none of us is as dumb as all of us.

Nobody is participating in Stack Overflow to make money. We’re participating in Stack Overflow because …

• We love programming
• We want to leave breadcrumb trails for other programmers to follow so they can avoid making the same dumb mistakes we did
• Teaching peers is one of the best ways to develop mastery
• We can follow our own interests wherever they lead
• We want to collectively build something great for the community with our tiny slices of effort

I don’t care how much you pay me, you’ll never be able to recreate the incredibly satisfying feeling I get when demonstrating mastery within my community of peers. That’s what we do on Stack Overflow: have fun, while making the internet one infinitesimally tiny bit better every day.

So is it any wonder that some claim Stack Overflow is more satisfying than their real jobs? Not to me.

If this all seems like a bunch of communist hippie bullcrap to you, I understand. It’s hard to explain. But there is quite a bit of science documenting these strange motivations. Let’s start with Dan Pink’s 2009 TED talk.

Dan’s talk centers on the candle problem. Given the following three items …

1. A candle
2. A box of thumbtacks
3. A book of matches

… how can you attach the candle to the wall?

It’s not a very interesting problem on its own — that is, until you try to incentivize teams to solve it:

Now I want to tell you about an experiment using the candle problem by a scientist from Princeton named Sam Glucksberg. Here’s what he did.

To the first group, he said, “I’m going to time you to establish norms, averages for how long it typically takes someone to solve this sort of problem.”

To the second group, he said, “If you’re in the top 25 percent of the fastest times you get five dollars. If you’re the fastest of everyone we’re testing here today you get 20 dollars.” (This was many years ago. Adjusted for inflation, it’s a decent sum of money for a few minutes of work.)

Question: How much faster did this group solve the problem?

Answer: It took them, on average, three and a half minutes longer. Three and a half minutes longer. Now this makes no sense, right? I mean, I’m an American. I believe in free markets. That’s not how it’s supposed to work. If you want people to perform better, you reward them. Give them bonuses, commissions, their own reality show. Incentivize them. That’s how business works. But that’s not happening here. You’ve got a monetary incentive designed to sharpen thinking and accelerate creativity — and it does just the opposite. It dulls thinking and blocks creativity.

It turns out that traditional carrot-and-stick incentives are only useful for repetitive, mechanical tasks. The minute you have to do anything even slightly complex that requires even a little problem solving without a clear solution or rules — those incentives not only don’t work, they make things worse!

Pink eventually wrote a book about this, Drive: The Surprising Truth About What Motivates Us.

There’s no need to read the book; this clever ten minute whiteboard animation will walk you through the main points. If you view only one video today, view this one.

The concept of intrinsic motivation may not be a new one, but I find that very few companies are brave enough to actually implement them.

I’ve tried mightily to live up to the ideals that Stack Overflow was founded on when building out my team. I don’t care when you come to work or what your schedule is. I don’t care where in the world you live (provided you have a great internet connection). I don’t care how you do the work. I’m not going to micromanage you and assign you a queue of task items. There’s no need.

If you want to build a ship, don’t drum up the men to gather wood, divide the work and give orders. Instead, teach them to yearn for the vast and endless sea.
– Antoine de Saint Exupéry

Because I know you yearn for the vast and endless sea, just like we do.

Go to Source

Well, I haven’t gone stir-crazy yet. I think. But in building Stack Overflow, I have learned a few things about what it means to work remotely — at least when it comes to programming. Our current team encompasses 5 people, distributed all over the USA, along with the team in NYC.

My first mistake was attempting to program alone. I had weekly calls with my business partner, Joel Spolsky, which were quite productive in terms of figuring out what it was we were trying to do together — but he wasn’t writing code. I was coding alone. Really alone. One guy working all by yourself alone. This didn’t work at all for me. I was unmoored, directionless, suffering from analysis paralysis, and barely able to get motivated enough to write even a few lines of code. I rapidly realized that I’d made a huge mistake in not having a coding buddy to work with.

that situation rectified itself soon enough, as I was fortunate enough to find one of my favorite old coding buddies was available. Even though Jarrod was in North Carolina and I was in California, the shared source code was the mutual glue that stuck us together, motivated us, and kept us moving forward. To be fair, we also had the considerable advantage of prior history, because we had worked together at a previous job. But the minimum bar to work remotely is to find someone who loves code as much as you do. It’s … enough. Anything else on top of that — old friendships, new friendships, a good working relationship — is icing that makes working together all the sweeter. I eventually expanded the team in the same way by adding another old coding buddy, Geoff, who lives in Oregon. And again by adding Kevin, who I didn’t know, but had built amazing stuff for us without even being asked to, from Texas. And again by adding Robert, in Florida, who I also didn’t know, but spent so much time on every single part of our sites that I felt he had been running alongside our team the whole way, there all along.

The reason remote development worked for us, in retrospect, wasn’t just shared love of code. I picked developers who I knew — I had incontrovertible proof — were amazing programmers. I’m not saying they’re perfect, far from it, merely that they were top programmers by any metric you’d care to measure. That’s why they were able to work remotely. Newbie programmers, or competent programmers who are phoning it in, are absolutely not going to have the moxie necessary to get things done remotely — at least, not without a pointy haired manager, or grumpy old team lead, breathing down their neck. Don’t even think about working remotely with anyone who doesn’t freakin’ bleed ones and zeros, and has a proven track record of getting things done.

While Joel certainly had a lot of high level input into what Stack Overflow eventually became, I only talked to him once a week, at best (these calls were the genesis of our weekly podcast series). I had a strong, clear vision of what I wanted Stack Overflow to be, and how I wanted it to work. Whenever there was a question about functionality or implementation, my team was able to rally around me and collectively make decisions we liked, and that I personally felt were in tune with this vision. And if you know me at all, you know I’m not shy about saying no, either. We were able to build exactly what we wanted, exactly how we wanted.

Bottom line, we were on a mission from God. And we still are.

So, there are a few basic ground rules for remote development, at least as I’ve seen it work:

• The minimum remote team size is two. Always have a buddy, even if your buddy is on another continent halfway across the world.
• Only grizzled veterans who absolutely love to code need apply for remote development positions. Mentoring of newbies or casual programmers simply doesn’t work at all remotely.
• If your remote team is going to work, they need full autonomy and a leader (PM, if you will) who has a strong vision and the power to fully execute on that vision.

This is all well and good when you have a remote team size of three, as we did for the bulk of Stack Overflow development. And all in the same country. Now we need to grow the company, and I’d like to grow it in distributed fashion, by hiring other amazing developers from around the world, many of whom I have met through Stack Overflow itself.

But how do you scale remote development? Joel had some deep seated concerns about this, so I tapped one of my heroes, Miguel de Icaza — who I’m proud to note is on our all-star board of advisors — and he was generous enough to give us some personal advice based on his experience running the Mono project, which has dozens of developers distributed all over the world.

At the risk of summarizing mercilessly (and perhaps too much), I’ll boil down Miguel’s advice the best I can. There are three tools you’ll need in place if you plan to grow a large-ish and still functional remote team:

1. Real time chat

   When your team member lives in Brazil, you can’t exactly walk by his desk to ask him a quick question, or bug him about something in his recent checkin. Nope. You need a way to casually ping your fellow remote team members and get a response back quickly. This should be low friction and available to all remote developers at all times. IM, IRC, some web based tool, laser beams, smoke signals, carrier pigeon, two tin cans and a string: whatever. As long as everyone really uses it.

   We’re currently experimenting with Campfire, but whatever floats your boat and you can get your team to consistently use, will work. Chat is the most essential and omnipresent form of communication you have when working remotely, so you need to make absolutely sure it’s functioning before going any further.

2. Persistent mailing list

   Sure, your remote team may know the details of their project, but what about all the other work going on? How do they find out about that stuff or even know it exists in the first place? You need a virtual bulletin board: a place for announcements, weekly team reports, and meeting summaries. This is where a classic old-school mailing list comes in handy.

   We’re using Google Groups and although it’s old school in spades, it works plenty well for this. You can get the emails as they arrive, or view the archived list via the web interface. One word of caution, however. Every time you see something arrive in your inbox from the mailing list you better believe, in your heart of hearts, that it contains useful information. The minute the mailing list becomes just another “whenever I have time to read that stuff”, noise engine, or distraction from work … you’ve let someone cry wolf too much, and ruined it. So be very careful. Noisy, argumentative, or useless things posted to the mailing list should be punishable by death. Or noogies.

3. Voice and video chat

   As much as I love ASCII, sometimes faceless ASCII characters just aren’t enough to capture the full intentions and feelings of the human being behind them. If you find yourself sending kilobytes of ASCII back and forth, and still are unsatisfied that you’re communicating, you should instill a reflexive habit of “going voice” on your team.

   Never underestimate the power of actually talking to another human being. I know, I know, the whole reason we got into this programming thing was to avoid talking to other people, but bear with me here. You can’t be face to face on a remote team without flying 6 plus hours, and who the heck has that kind of time? I’ve got work I need to get done! Well, the next best thing to hopping on a plane is to fire up Skype and have a little voice chat. Easy peasy. All that human nuance which is totally lost in faceless ASCII characters (yes, even with our old pal *<:-)) will come roaring back if you regularly schedule voice chats. I recommend at least once a week at an absolute minimum; they don’t have to be long meetings, but it sure helps in understanding the human being behind all those awesome checkins.
   

Nobody hates meetings and process claptrap more than I do, but there is a certain amount of process you’ll need to keep a bunch of loosely connected remote teams and developers in sync.

1. Monday team status reports

   Every Monday, as in somebody’s-got-a-case-of-the, each team should produce a brief, summarized rundown of:

   • What we did last week
   • What we’re planning to do this week
   • Anything that is blocking us or we are concerned about

   This doesn’t have to be (and in fact shouldn’t be) a long report. The briefer the better, but do try to capture all the useful highlights. Mail this to the mailing list every Monday like clockwork. Now, how many “teams” you have is up to you; I don’t think this needs to be done at the individual developer level, but you could.

2. Meeting minutes

   Any time you conduct what you would consider to be a “meeting” with someone else, take minutes! That is, write down what happened in bullet point form, so those remote team members who couldn’t be there can benefit from — or at least hear about — whatever happened.

   Again, this doesn’t have to be long, and if you find taking meeting minutes onerous then you’re probably doing it wrong. A simple bulleted list of sentences should suffice. We don’t need to know every little detail, just the big picture stuff: who was there? What topics were discussed? What decisions were made? What are the next steps?

Both of the above should, of course, be mailed out to the mailing list as they are completed so everyone can be notified. You do have a mailing list, right? Of course you do!

If this seems like a lot of jibba-jabba, well, that’s because remote development is hard. It takes discipline to make it all work, certainly more discipline than piling a bunch of programmers into the same cubicle farm. But when you imagine what this kind of intellectual work — and by that I mean not just programming, but anything where you’re working in mostly thought-stuff — will be like in ten, twenty, even thirty years … don’t you think it will look a lot like what happens every day right now on Stack Overflow? That is, a programmer in Brazil helping a programmer in New Jersey solve a problem?

If I have learned anything from Stack Overflow it is that the world of programming is truly global. I am honored to meet these brilliant programmers from every corner of the world, even if only in a small way through a website. Nothing is more exciting for me than the prospect of adding international members to the Stack Overflow team — so that the development of Stack Overflow more properly reflects what Stack Overflow is: an international effort of like-minded — and dare I say totally awesome — programmers. If I could I’d hire each and every one of you. OK, maybe I’m a little biased. But to me, that’s how awesome the Stack Overflow community is.

In many ways, remote development represents the future of work. If we have to spend a little time figuring out how this stuff works, and maybe even make some mistakes along the way, it’s worth it. As far as I’m concerned, the future is now. Why wait?

Go to Source

• Server Fault
• Super User
• Stack Overflow
• Meta Stack Overflow

(In case you were wondering, yes, meta is the Star Wars Holiday Special.)

These sites all run the same core engine, but the logo, domain, and CSS “skin” that lies over the HTML skeleton is different in each case:

They are not terribly different looking, it’s true, but we also want them to be recognizable as a family of sites.

We’re working with two amazing designers, Jin Yang and Nathan Bowers, who are helping us whip the CSS and HTML into shape so they can produce a set of about 10 different Zen Garden designs. As new sites in our network get democracied into being, these designs will be used as a palette for the community to choose from. (And, later, the community will decide on a domain name and logo as well.)

Anyway, I bring this up not because my pokemans, let me show you them, but because I have to personally maintain four different CSS files. And that number is only going to get larger. Much larger. That scares me a little.

Most of all, what I’ve learned from this exercise in site theming is that CSS is kind of painful. I fully support CSS as a (mostly) functional user interface Model-View-Controller. But even if you have extreme HTML hygiene and Austrian levels of discipline, CSS has some serious limitations in practice.

Things in particular that bite us a lot:

• Vertical alignment is a giant, hacky PITA. (Tables work great for this though!)
• Lack of variables so we have to repeat colors all over the place.
• Lack of nesting so we have to repeat huge blocks of CSS all over the place.

In short, CSS violates the living crap out of the DRY principle. You are constantly and unavoidably repeating yourself.

That’s why I’m so intrigued by two Ruby gems that attempt to directly address the deficiencies of CSS.

1. Less CSS

/* CSS */

#header {
  -moz-border-radius: 5;
  -webkit-border-radius: 5;
  border-radius: 5;
}

#footer {
  -moz-border-radius: 10;
  -webkit-border-radius: 10;
  border-radius: 10;
}

// LessCSS

.rounded_corners (@radius: 5px) {
  -moz-border-radius: @radius;
  -webkit-border-radius: @radius;
  border-radius: @radius;
}

#header {
  .rounded_corners;
}

#footer {
  .rounded_corners(10px);
}

2. SASS

/* CSS */

.content_navigation {
  border-color: #3bbfce;
  color: #2aaebd;
}

.border {
  padding: 8px;
  margin: 8px;
  border-color: #3bbfce;
}

// Sass

!blue = #3bbfce
!margin = 16px

.content_navigation
  border-color = !blue
  color = !blue - #111

.border
  padding = !margin / 2
  margin = !margin / 2
  border-color = !blue

As you can see, in both cases we’re transmogrifying CSS into a bit more of a programming language, rather than the static set of layout rules it currently exists as. Behind the scenes, we’re generating plain vanilla CSS using these little dynamic languages. This could be done at project build time, or even dynamically on every page load if you have a good caching strategy.

I’m not sure how many of these improvements CSS3 will bring, never mind when the bulk of browsers in the world will support it. But I definitely feel that the core changes identified in both Less CSS and SASS address very real pain points in practical CSS use. It’s worth checking them out to understand why they exist, what they bring to the table, and how you could possibly adopt some of these strategies in your own CSS and your favorite programming language.

Go to Source

Despite my misgivings, email is the cockroach of communication mediums: you just can’t kill it. Email is the one method of online contact that almost everyone — at least for that subset of “everyone” which includes people who can bear to touch a computer at all — is guaranteed to have, and use. Yes, you can make a fairly compelling case that email is for old stupid people, but let’s table that discussion for now.

So, reluctantly, we come to the issue of sending email through code. It’s easy! Let’s send some email through oh, I don’t know, let’s say … Ruby, courtesy of some sample code I found while browsing the Ruby tag on Stack Overflow.

require 'net/smtp'

def send_email(to, subject = "", body = "")
    from = "my@email.com"
    body= "From: #{from}\r\nTo: #{to}\r\nSubject: #{subject}\r\n\r\n#{body}\r\n"

    Net::SMTP.start('192.168.10.213', 25, '192.168.0.218') do |smtp|
        smtp.send_message body, from, to
    end
end

send_email "foo@example.com", "title", "body goes here"

There’s a bug in this code, though. Do you see it?

Just because you send an email doesn’t mean it will arrive. Not by a long shot. Bear in mind this is email we’re talking about. It was never designed to survive a bitter onslaught of criminals and spam, not to mention the explosive, exponential growth it has seen over the last twenty years. Email is a well that has been truly and thoroughly poisoned — the digital equivalent of a superfund cleanup site. The ecosystem around email is a dank miasma of half-implemented, incompletely supported anti-spam hacks and workarounds.

Which means the odds of that random email your code just sent getting to its specific destination is .. spotty. At best.

If you want email your code sends to actually arrive in someone’s AOL mailbox, to the dulcet tones of “You’ve Got Mail!”, there are a few things you must do first. And most of them are only peripherally related to writing code.

1. Make sure the computer sending the email has a Reverse PTR record

What’s a reverse PTR record? It’s something your ISP has to configure for you — a way of verifying that the email you send from a particular IP address actually belongs to the domain it is purportedly from.

Not every IP address has a corresponding PTR record. In fact, if you took a random sampling of addresses your firewall blocked because they were up to no good, you’d probably find most have no PTR record – a dig -x gets you no information. That’s also apt to be true for mail spammers, or their PTR doesn’t match up: if you do a dig -x on their IP you get a result, but if you look up that result you might not get the same IP you started with.

That’s why PTR records have become important. Originally, PTR records were just intended as a convenience, and perhaps as a way to be neat and complete. There still are no requirements that you have a PTR record or that it be accurate, but because of the abuse of the internet by spammers, certain conventions have grown up. For example, you may not be able to send email to some sites if you don’t have a valid PTR record, or if your pointer is “generic”.

How do you get a PTR record? You might think that this is done by your domain registrar – after all, they point your domain to an IP address. Or you might think whoever handles your DNS would do this. But the PTR record isn’t up to them, it’s up to the ISP that “owns” the IP block it came from. They are the ones who need to create the PTR record.

A reverse PTR record is critical. How critical? Don’t even bother reading any further until you’ve verified that your ISP has correctly configured the reverse PTR record for the server that will be sending email. It is absolutely the most common check done by mail servers these days. Fail the reverse PTR check, and I guarantee that a huge percentage of the emails you send will end up in the great bit bucket in the sky — and not in the email inboxes you intended.

2. Configure DomainKeys Identified Mail in your DNS and code

What’s DomainKeys Identified Mail? With DKIM, you “sign” every email you send with your private key, a key only you could possibly know. And this can be verified by attempting to decrypt the email using the public key stored in your public DNS records. It’s really quite clever!

The first thing you need to do is generate some public-private key pairs (one for every domain you want to send email from) via OpenSSL. I used a win32 version I found. Issue these commands to produce the keys in the below files:

$ openssl genrsa -out rsa.private 1024
$ openssl rsa -in rsa.private -out rsa.public -pubout -outform PEM

These public and private keys are just big ol’ Base64 encoded strings, so plop them in your code as configuration string resources that you can retrieve later.

Next, add some DNS records. You’ll need two new TXT records.

1. _domainkey.example.com
   “o=~\; r=contact@example.com”

2. selector._domainkey.example.com
   “k=rsa\; p={public-key-base64-string-here}”

The first TXT DNS record is the global DomainKeys policy and contact email.

The second TXT DNS record is the public base64 key you generated earlier, as one giant unbroken string. Note that the “selector” part of this record can be anything you want; it’s basically just a disambiguating string.

Almost done. One last thing — we need to sign our emails before sending them. In any rational world this would be handled by an email library of some kind. We use Mailbee.NET which makes this fairly painless:

smtp.Message = dk.Sign(smtp.Message,
               null, AppSettings.Email.DomainKeyPrivate, false, "selector");

3. Set up a SenderID record in your DNS

To be honest, SenderID is a bit of a “nice to have” compared to the above two. But if you’ve gone this far, you might as well go the distance. SenderID, while a little antiquated and kind of.. Microsoft/Hotmail centric.. doesn’t take much additional effort.

SenderID isn’t complicated. It’s another TXT DNS record at the root of, say, example.com, which contains a specially formatted string documenting all the allowed IP addresses that mail can be expected to come from. Here’s an example:

"v=spf1 a mx ip4:10.0.0.1 ip4:10.0.0.2 ~all"

You can use the Sender ID SPF Record Wizard to generate one of these for each domain you send email from.

That sucked. How do I know all this junk is working?

I agree, it sucked. Email sucks; what did you expect? I used two methods to verify that all the above was working:

1. Test emails sent to a GMail account.

   Use the “show original” menu on the arriving email to see the raw message content as seen by the email server. You want to verify that the headers definitely contain the following:

   Received-SPF: pass
   Authentication-Results: ... spf=pass ... dkim=pass
   

   If you see that, then the Reverse PTR and DKIM signing you set up is working. Google provides excellent diagnostic feedback in their email server headers, so if something isn’t working, you can usually discover enough of a hint there to figure out why.

2. Test emails sent to the Port25 email verifier

   Port25 offers a really nifty public service — you can send email to check-auth@verifier.port25.com and it will reply to the from: address with an extensive diagnostic! Here’s an example summary result from a test email I just sent to it:

   SPF check:          pass
   DomainKeys check:   fail
   DKIM check:         pass
   Sender-ID check:    pass
   SpamAssassin check: ham
   

   You want to pass SPF, DKIM, and Sender-ID. Don’t worry about the DomainKeys failure, as I believe it is spurious — DKIM is the “newer” version of that same protocol.

Yes, the above three steps are quite a bit of work just to send a lousy email. But I don’t send email lightly. By the time I’ve reached the point where I am forced to write code to send out email, I really, really want those damn emails to arrive. By any means necessary.

And for those who are the unfortunate recipients of these emails: my condolences.

Go to Source

• Multiple Monitors and Productivity
• Multiple LCDs
• Joining the Prestigious Three Monitor Club
• The Large Display Paradox
• LCD Monitor Arms

If you’re a long time reader you’re probably sick of hearing about this stuff by now, but something rather wonderful has happened since I last wrote about it:

If you’re only using one monitor, you are cheating yourself out of potential productivity. Two monitors is a no-brainer. It’s so fundamental that I included it as a part of the Programmer’s Bill of Rights.

But you can do better.

As good as two monitors is, three monitors is even better. With three monitors, there’s a “center” to focus on. And 50% more display area. While there’s certainly a point of diminishing returns for additional monitors, I think three is the sweet spot. Even Edward Tufte, in the class I recently attended, explicitly mentioned multiple monitors. I don’t care how large a single display can be; you can never have enough desktop space.

Normally, to achieve three monitors, you have to either:

1. Buy an exotic video card that has more than 2 monitor connections.
2. Install a second video card.

Fortunately, that is no longer true. I was excited to learn that the latest ATI video cards have gone from two to three video outputs. Which means you can now achieve triple monitors with a single video card upgrade! They call this “eyefinity”, but it’s really just shorthand for “raising the standard from two display outputs to three”.

But, there is a (small) catch. The PC ecosystem is in the middle of shifting display output standards. For evidence of this, you need look no further than the back panel of one of these newfangled triple display capable ATI video cards:

It contains:

• two DVI outputs
• one HDMI output
• one DisplayPort output

I suspect part of this odd connector layout is due to space restrictions (DVI is awfully chunky), but I’ve always understood DisplayPort to be the new, improved DVI connector for computer monitors, and HDMI to be the new, improved s-video/component connector for televisions. Of course these worlds are blurring, as modern high-definition TVs make surprisingly effective computer monitors, too.

Anyway, since all my monitors have only DVI inputs, I wasn’t sure what to do with the other output. So I asked on Super User. The helpful answers led me to discover that, as I suspected, the third output has to be DisplayPort. So to connect my third monitor, I needed to convert DisplayPort to DVI, and there are two ways:

1. a passive, analog DisplayPort to DVI conversion cable for ~$30 that supports up to 1920×1200
2. an active, digital DisplayPort to DVI converter for $110 that supports all resolutions

I ended up going with the active converter, which has mixed reviews, but it’s worked well for me over the last few weeks.

Note that this adapter requires USB power, and given the spotty results others have had with it, some theorize that it needs quite a bit of juice to work reliably. I plugged it into my system’s nearby rear USB ports which do tend to deliver more power (they’re closer to the power supply, and have short cable paths). Now, I have gotten the occasional very momentary black screen with it, but nothing severe enough to be a problem or frequent enough to become a pattern. If you have DisplayPort compatible monitors, of course, this whole conversion conundrum is a complete non-issue. But DisplayPort is fairly new, and even my new-ish LCD monitors don’t support it yet.

The cool thing about this upgrade, besides feeding my video card addiction, is that I was able to simplify my hardware configuration. That’s always good. I went from two video cards to one, which means less power consumption, simpler system configuration, and fewer overall driver oddities. Basically, it makes triple monitors — dare I say it — almost a mainstream desktop configuration. How could I not be excited about that?

I was also hoping that Nvidia would follow ATI’s lead here and make three display outputs the standard for all their new video cards, too, but sadly that’s not the case. It turns out their new GTX 480 fails in other ways, in that it’s basically the Pentium 4 of video cards — generating ridiculous amounts of heat for very little performance gain. Based on those two facts, I am comfortable endorsing ATI wholeheartedly at this point. But, do be careful, because not all ATI cards support triple display outputs (aka “eyefinity”). These are the ones that I know do:

• Radeon HD 5670 (~$100)
• Radeon HD 5770 (~$150)
• Radeon HD 5830 (~$250)
• Radeon HD 5850 (~$320)
• Radeon HD 5870 (~$450)

Unless you’re a gamer, there’s no reason to care about anything other than the least expensive model here, which will handily crush any 2D or 3D desktop GUI acceleration needs you might have. As an addict, of course I bought the high end model and it absolutely did not disappoint — more than doubling my framerates in the excellent game Battlefield: Bad Company 2 over the GTX 280 I had before.

I’m excited that a triple monitor setup is now, thanks to ATI, so easily attainable for desktop users — as long as you’re aware of the DisplayPort caveat I discussed above. I’d encourage anyone who is even remotely interested in the (many) productivity benefits of a triple monitor setup to seriously consider an ATI video card upgrade.

Go to Source

I’ve been a long time fan of Krug's book Don't Make Me Think. Not just because it’s a quick, easy read (and it is!) — but because it’s the most concise and most approachable book I’ve ever found to teach the fundamental importance of usability. As far as I’m concerned, if you want to help us make the software industry a saner place, the first step is getting Don't Make Me Think in the hands of as many of your coworkers as you can. If you don’t have people that care about usability on your project, your project is doomed.

Beyond getting people over the hurdle of at least paging through the Krug book, and perhaps begrudgingly conceding that this usability stuff matters, the next challenge is figuring out how to integrate usability testing into your project. It’s easy to say “Usability is Important!”, but you have to walk the walk, too. I touched on some low friction ways to get started in Low-Fi Usability Testing. That rough outline is now available in handy, more complete book form — Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing Usability Problems.

Don’t worry, Krug’s book is just as usable as his advice. It’s yet another quick, easy read. Take it from the man himself:

• Usability testing is one of the best things people can do to improve Web sites (or almost anything they’re creating that people have to interact with).
• Since most organizations can’t afford to hire someone to do testing for them
  on a regular basis, everyone should learn to do it themselves. And …

• I could probably write a pretty good book explaining how to do it.

If you’re wondering what the beginner’s “how do I boil water?” recipe for software project usability is, stop reading this post and get a copy of Rocket Surgery Made Easy. Now.

One of the holy grails of usability testing is eyetracking — measuring where people’s eyes look as they use software and web pages. Yes, there are clever JavaScript tools that can measure where users move their pointers, but that’s only a small part of the story. Where the eye wanders, the pointer may not, and vice-versa. But, who has the time and equipment necessary to conduct an actual eyetracking study? Almost nobody.

That’s where Eyetracking Web Usability comes in.

Eyetracking Web Usability is chock full of incredibly detailed eyetracking data for dozens of websites. Even though you (probably) can’t afford to do real eyetracking, you can certainly use this book as a reference. There is enough variety in UI and data that you can map the results, observations, and explanations found here to what your project is doing.

This particular book is rather eyetracking specific, but it’s just the latest entry in a whole series on usability, and I recommend them all highly. These books are a fount of worthwhile data for anyone who works on software and cares about usability, from one of the most preeminent usability experts on the web.

Usability isn’t really cheap or easy. It’s an endless war, with innumerable battlegrounds, stretching all the way back to the dawn of computing. But these books, at least, are cheap and easy in the sense that they give you some basic training in fighting the good (usability) fight. That’s the best I can do, and it’s all I’d ask from anyone else I work with.

Go to Source

The short version of Fitts’ law, to save you all that tedious reading, is this:

• Put commonly accessed UI elements on the edges of the screen. Because the cursor automatically stops at the edges, they will be easier to click on.
• Make clickable areas as large as you can. Larger targets are easier to click on.

I know, it’s very simple, almost too simple, but humor me by following along with some thought exercises. Imagine yourself trying to click on …

• a 1 x 1 target at a random location
• a 5 x 5 target at a random location
• a 50 x 50 target at a random location
• a 5 x 5 target in the corner of your screen
• a 1 x 100 target at the bottom of your screen

Fitts’ Law is mostly common sense, and enjoys enough currency with UI designers that they’re likely to know about it even if they don’t follow it as religiously as they should. Unfortunately, I’ve found that designers are much less likely to consider the opposite of Fitts’ Law, which is arguably just as important.

If we should make UI elements we want users to click on large, and ideally place them at corners or edges for maximum clickability — what should we do with UI elements we don’t want users to click on? Like, say, the “delete all my work” button?

Alan Cooper, in About Face 3, calls this the ejector seat lever.

In the cockpit of every jet fighter is a brightly painted lever that, when pulled, fires a small rocket engine underneath the pilot’s seat, blowing the pilot, still in his seat,
out of the aircraft to parachute safely to earth. Ejector seat levers can only be used
once, and their consequences are significant and irreversible.

Applications must have ejector seat levers so that users can—occasionally—move
persistent objects in the interface, or dramatically (sometimes irreversibly) alter the function or behavior of the application. The one thing that must never happen is accidental deployment of the ejector seat.

The interface design must assure that a user can never inadvertently fire the ejector seat when all he wants to do is make some minor adjustment to the program.

I can think of a half-dozen applications I regularly use where the ejector seat button is inexplicably placed right next to the cabin lights button. Let’s take a look at our old friend GMail, for example:

I can tell what you’re thinking. Did he click Send or Save Now? Well, to tell you the truth, in all the excitement of composing that angry email, I kind of lost track myself. Good thing we can easily undo a sent mail! Oh wait, we totally can’t. Consider my seat, or at least that particular rash email, ejected.

It’s even worse when I’m archiving emails.

While there were at least 10 pixels between the buttons in the previous example, here there are all of … three. Every few days I accidentally click Report Spam when I really meant to click Archive. Now, to Google’s credit, they do offer a simple, obvious undo path for these accidental clicks. But I can’t help wondering why it is, exactly, that these two buttons with such radically different functionality just have to be right next to each other.

Undo is powerful stuff, but wouldn’t it be better still if I wasn’t pulling the darn ejector seat lever all the time? Wouldn’t it make more sense to put that risky ejector seat lever in a different location, and make it smaller? Consider the WordPress post editor.

Here, the common Update operation is large and obviously a button — it’s easy to see and easy to click on. The less common Move to Trash operation is smaller, presented as a vanilla hyperlink, and placed well away from Update.

The next time you’re constructing a user interface, you should absolutely follow Fitts’ law. It just makes sense. But don’t forget to follow the opposite of Fitts’ law, too — uncommon or dangerous UI items should be difficult to click on!

Go to Source

If you are building an ASP.NET web application that’s going to get thousands of hits per hour, the execution overhead of Linq queries is going to consume too much CPU and make your site slow. There’s a runtime cost associated with each and every Linq Query you write. The queries are parsed and converted to a nice SQL Statement on every hit. It’s not done at compile time because there’s no way to figure out what you might be sending as the parameters in the queries during runtime.

So, if you have common Linq to Sql statements like the following one ..

var query = from widget in dc.Widgets
            where widget.ID == id && widget.PageID == pageId
            select widget;
var widget = query.SingleOrDefault();

.. throughout your growing web application, you are soon going to have scalability nightmares.

J.D. Conley goes further:

So I dug into the call graph a bit and found out the code causing by far the most damage was the creation of the LINQ query object for every call! The actual round trip to the database paled in comparison.

I must admit, these results seem … unbelievable. Querying the database is so slow (relative to typical code execution) that if you have to ask how long it will take, you can’t afford it. I have a very hard time accepting the idea that dynamically parsing a Linq query would take longer than round-tripping to the database. Pretend I’m from Missouri: show me. Because I am unconvinced.

All of this is very curious, because Stack Overflow uses naive, uncompiled Linq queries on every page, and we are a top 1,000 website on the public internet by most accounts these days. We get a considerable amount of traffic; the last time I checked it was about 1.5 million pageviews per day. We go to great pains to make sure everything is as fast as we can. We’re not as fast as we’d like to be yet, but I think we’re doing a reasonable job so far. The journey is still very much underway — we realize that overnight success takes years.

Anyway, Stack Overflow has dozens to hundreds of plain vanilla uncompiled Linq to SQL queries on every page. What we don’t have is “scalability nightmares”. CPU usage has been one of our least relevant constraints over the last two years as the site has grown. We’ve also heard from other development teams, multiple times, that Linq to SQL is “slow”. But we’ve never been able to reproduce this even when armed with a profiler.

Quite the mystery.

Now, it’s absolutely true that Linq to SQL has the performance peculiarity both posters are describing. We know that’s true because Rico tells us so, and Rico … well, Rico’s the man.

In short the problem is that the basic Linq construction (we don’t really have to reach for a complex query to illustrate) results in repeated evaluations of the query if you ran the query more than once.

Each execution builds the expression tree, and then builds the required SQL. In many cases all that will be different from one invocation to another is a single integer filtering parameter. Furthermore, any databinding code that we must emit via lightweight reflection will have to be jitted each time the query runs. Implicit caching of these objects seems problematic because we could never know what good policy is for such a cache — only the user has the necessary knowledge.

It’s fascinating stuff; you should read the whole series.

What’s unfortunate about Linq in this scenario is that you’re intentionally sacrificing something that any old and busted SQL database gives you for free. When you send a garden variety parameterized SQL query through to a traditional SQL database, it’s hashed, then matched against existing cached query plans. The computational cost of pre-processing a given query is only paid the first time the database sees the new query. All subsequent runs of that same query use the cached query plan and skip the query evaluation. Not so in Linq to SQL. As Rico said, every single run of the Linq query is fully parsed every time it happens.

Now, there is a way to compile your Linq queries, but I personally find the syntax kind of … ugly and contorted. You tell me:

Func<Northwinds, IQueryable<Orders>, int> q =
     CompiledQuery.Compile<Northwinds, int, IQueryable<Orders>>
                ((Northwinds nw, int orderid) =>
                            from o in nw.Orders
                            where o.OrderId == orderid
                            select o );

Northwinds nw = new Northwinds(conn);

foreach (Orders o in q(nw, orderid))
{
}

Anyway, that’s neither here nor there; we can confirm the performance penalty of failing to compile our queries ourselves. We recently wrote a one time conversion job against a simple 3 column table containing about 500,000 records. The meat of it looked like this:

db.PostTags.Where(t => t.PostId == this.Id).ToList();

Then we compared it with the SQL variant; note that this is also being auto-cast down to the handy PostTag object as well, so the only difference is whether or not the query itself is SQL.

db.ExecuteQuery(
   "select * from PostTags where PostId={0}", this.Id).ToList();

On an Intel Core 2 Quad running at 2.83 GHz, the former took 422 seconds while the latter took 275 seconds.

The penalty for failing to compile this query, across 500k iterations, was 147 seconds. Wow! That’s 1.5 times slower! Man, only a BASIC programmer would be dumb enough to skip compiling all their Linq queries. But wait a second, no, wait 147 seconds. Let’s do the math, even though I suck at it. Each uncompiled run of the query took less than one third of a millisecond longer.

At first I was worried that every Stack Overflow page was 1.5 times slower than it should be. But then I realized it’s probably more realistic to make sure that any page we generate isn’t doing 500 freakin’ thousand queries! Have we found ourselves in the sad tragedy of micro-optimization theater … again? I think we might have. Now I’m just depressed.

While it’s arguably correct to say that every compiled Linq query (or for that matter, any compiled anything) will be faster, your decisions should be a bit more nuanced than compiled or bust. How much benefit you get out of compilation depends how many times you’re doing it. Rico would be the first to point this out, and in fact he already has:

Testing 1 batches of 5000 selects 


uncompiled  543.48 selects/sec
compiled    925.75 selects/sec


Testing 5000 batches of 1 selects 


uncompiled  546.03 selects/sec
compiled    461.89 selects/sec

Have I mentioned that Rico is the man? Do you see the inversion here? Either you’re doing 1 batch of 5000 queries, or 5000 batches of 1 query. One is dramatically faster when compiled; the other is actually a big honking net negative if you consider the developer time spent converting all those beautifully, wonderfully simple Linq queries to the contorted syntax necessary for compilation. Not to mention the implied code maintenance.

I’m a big fan of compiled languages. Even Facebook will tell you that PHP is about as half as fast as it should be on a good day with a tailwind. But compilation alone is not the entire performance story. Not even close. If you’re compiling something — whether it’s PHP, a regular expression, or a Linq query, don’t expect a silver bullet, or you may end up disappointed.

Go to Source

The article Why Can’t Programmers… Program? changed the way I did interviews. I used to lead off by building rapport. That proved to be too time-consuming when, as you mentioned, the vast majority of candidates were simply non-technical. So I started leading off with technical questions. First progressing from easy to hard questions. Then I noticed I identified the rejects faster if I went the other way – hard questions first – so long as the hard questions were still in the “if you don’t know this then you can’t work here” category. Most of my interviews still took about twenty minutes, because tough questions take some time to answer and evaluate. But it was a big improvement over the rapport-building method; and it could be done over the phone.

After reading your article, I started doing code interviews over the phone, using web meetings. My interview times were down to about 15 minutes each to identify people who just can’t code— the vast majority.

I wrote that article in 2007, and I am stunned, but not entirely surprised, to hear that three years later “the vast majority” of so-called programmers who apply for a programming job interview are unable to write the smallest of programs. To be clear, hard is a relative term — we’re not talking about complicated, Google-style graduate computer science interview problems. This is extremely simple stuff we’re asking candidates to do. And they can’t. It’s the equivalent of attempting to hire a truck driver and finding out that 90 percent of the job applicants can’t find the gas pedal or the gear shift.

I agree, it’s insane. But it happens every day, and is (apparently) an epidemic hiring problem in our industry.

You have to get to the simple technical interview questions immediately to screen out the legions of non-programming programmers. Screening over the telephone is a wise choice, as I’ve noted before. But screening over the internet is even better, and arguably more natural for code.

I still wasn’t super-happy with having to start up the web meeting and making these guys share their desktops with me. I searched for other suitable tools for doing short “pen-and-paper” style coding interviews over the web, but I couldn’t find any. So I did what any self-respecting programmer would do. I wrote one.

Man, was it worth it! I schedule my initial technical screenings with job applicants in 15-minute blocks. I’m usually done in 5-10 minutes, sadly. I schedule an actual interview with them if they can at least write simple a 10-line program. That doesn’t happen often, but at least I don’t have to waste a whole lot of time anymore.

Mike adds a disclaimer that his homegrown coding interview tool isn’t meant to show off his coding prowess. He needed a tool, so he wrote one — and thoughtfully shared it with us. There might well be others out there; what online tools do you use to screen programmers?

Three years later, I’m still wondering: why do people who can’t write a simple program even entertain the idea they can get jobs as working programmers? Clearly, some of them must be succeeding. Which means our industry-wide interviewing standards for programmers are woefully inadequate, and that’s a disgrace. It’s degrading to every working programmer.

At least bad programmers can be educated; non-programming programmers are not only hopeless but also cheapen the careers of everyone around them. They must be eradicated, starting with simple technical programming tests that should be a part of every programmer interview.

Go to Source

1. Continuing fallout from International Backup Awareness Day, which meant all updates to Coding Horror from that point onward were hand-edited text files. Which, believe me, isn’t nearly as sexy as it … uh … doesn’t sound.

2. I am presenting and conducting a workshop at Webstock 2010 in New Zealand. This is a two week trip I’m taking with the whole family, including our little buddy Rock Hard Awesome, so the preparations have been more intense than usual.

   On top of all that, according to the program, I just found that my presentation involves interpretive dance, too. Man. I wish someone had told me! My moves are so rusty, they’ve barely improved from Electric Boogaloo. But hey, at least I don’t have to sing Andrews Sister songs like poor Brian Fling.

And then, of course, there’s that crazy Stack Overflow thing I’m always yammering on about. Very busy there, our team is expanding, and we have big plans for this year, too.

But, there is hope!

Thanks to the fine folks at Six Apart — and more specifically the herculean efforts of one Michael Sippey — Coding Horror is now hosted in the TypePad ecosystem. Which means, at least in theory, better “cloud” type reliability in the future. (cough)

One accidental bit of collateral damage was that comments, by necessity, were disabled during this two month period. At first, I was relieved. This may seem a bit hypocritical, since I originally wrote A Blog Without Comments is Not a Blog. And I still believe it too. But as I prophetically noted in the very same post:

I am sympathetic to issues of scale. Comments don’t scale worth a damn. If you have thousands of readers and hundreds of comments for every post, you should disable comments and switch to forums, probably moderated forums at that. But the number of bloggers who have that level of readership is so small as to be practically nil. And when you get there, believe me, you’ll know. Until then, you should enable comments.

I guess you can put this in the “nice problems to have” category, but let me tell you, it’s not so nice of a problem when it’s on your plate. At a large enough scale, comments require active moderation or they rapidly go sour. People get mean, the crazies come out in full force, and the comments start to resemble an out of control trailer park reality show brawl. It’s fun, I suppose, but in a way that drives out all the sane people. Left unchecked, the best you can hope for is to end up head resident at the sanitarium. And that’s a hell of a way to go out.

(the above is from Mike Reed’s amazing Flame Warriors series, by the way. Well worth your time if you haven’t seen it already.)

The degeneration of comments was a shame, because it undermined my claim that comments are awesome.

It’s an open secret amongst bloggers that the blog comments are often better than the original blog post, and it’s because the community collectively knows far more than you or I will ever know.

The best part of a blog post often begins where the blog post ends. If you are offended by that, I humbly submit you don’t understand why blogs work.

Why would I have bothered to found Stack Overflow with Joel Spolsky if I didn’t believe in the power of community — that none of us is as dumb as all of us? Honestly, a lot of the design of Stack Overflow comes from my personal observations about how blog comments work. But my creaky old Coding Horror comments offered none of the fancy voting and moderation facilities that make Stack Overflow work. And without ample free personal time and attention from me to weed the comment garden, the comments got out of control.

Most of all, I blame myself.

I got some amazing emails in lieu of comments on my last few blog posts, and it positively kills me that these emails were only seen by two sets of eyes instead of the thousands they deserve. That’s a big part of why I hate email silos. And really, email in general.

But there was another unanticipated side effect of having comments disabled that Stéphane Charette pointed out to me in email.

Here is an interesting “silver lining” to the crash you had. Without
comments, it forces us, your faithful readers, to think more about
what you have to say.

In a way, things are back to how your blog used
to be. In recent years, the huge influx of comments means that we —
or just I? — end up spending 1/4 of my time reading what you wrote,
and then merging in what everyone else wrote. Depending on how I feel
about the topic and your approach to the issue, the weight values may
be very different than 50/50. But regardless, I always have to
consider when clicking on my Coding Horror bookmark: “Is now the
right time to check if he has a new entry? Do I have enough time to
read through a hundred comments? Should I wait until later tonight
when the kids are in bed to go read his latest article?”

I never thought about it until recently. Your crash is what brought
this up to light. Like tonight, when I saw your new headline in my
iGoogle page, I didn’t have to consider whether or not it was the
right time. I read the article, and then thought for myself. I
didn’t let other people’s comments steer my thoughts. How nice!

I’m not certain why it works like this. Often, the sheer number of
comments distracts from what you wrote, but for some reason, it is
impossible not to at the very least scroll through what people say.
In a way, your blog has ended up like a slashdot article, with a
paragraph or two of content at the top, and then everyone wanting to
insert their $0.02.

Thinking for yourself. Now there’s a novel idea. In the reverberating echo chamber that is the internet, I think we would all do well to remind ourselves of that periodically.

He’s also right that the psychic burden of all those comments was weighing not just on readers, but on me, the writer, too. That’s why I had a false sense of freedom when comments were disabled. You mean I can say whatever I want, and nobody can contradict me underneath my very own post? Revolutionary!

There are some absolute gems of insight and observation in comments, but sometimes extracting them was too much like pulling teeth. At the same time, I felt obligated to read all the comments on every post of mine. If I was asking people to read the random words I’m spewing all over the internet, how could I not extend my commenters the same courtesy? That’s just rude.

It seems the only thing worse than comments being on was comments being off. It started to feel empty. As if I was in an enormous room, presenting to an eerily mute audience.

So, while I am very glad to have comments back, and I welcome dialog with the community, there will be … changes. For the benefit of everyone’s mental health.

1. No more anonymous comments. While I would prefer to allow anonymous comments, it is clear that at this scale I don’t have time to deal properly with anonymous comments. If you want to say something, you’ll need to authenticate. If what you have to say isn’t worth authenticating to post, it’s probably best for both of us if you keep it to yourself anyway.

   The good news is that the TypePad commenting system supports a veritable laundry list of authentication mechanisms — OpenID (naturally), Twitter, Facebook, Google, Yahoo, and many others. So authenticating to post a comment should only present a mild, but necessary, barrier to conversation.

2. Comment moderation will be more stringent. If you don’t have something useful and reasonably constructive to say in your comment, it will be removed without hesitation. You can be as critical of me (or, better still, my arguments and ideas) as you like, but you must convince me that you’re contributing to the conversation and not just yelling at me or anyone else.

   I’m not looking for sycophants, but shrill argument is every bit as bad. When you comment here, try to show the class something interesting they can use. That’s all I’m asking.

It feels good to be back. Thanks to Six Apart for making it happen.

And, most of all, thanks to you for reading.

Go to Source

It's so funny when I hear people being so protective of ideas. (People who want me to sign an NDA to tell me the simplest idea.) To me, ideas are worth nothing unless executed. They are just a multiplier. Execution is worth millions.

To make a business, you need to multiply the two. The most brilliant idea, with no execution, is worth $20. The most brilliant idea takes great execution to be worth $20,000,000. That's why I don't want to hear people's ideas. I'm not interested until I see their execution.

I was reminded of Mr. Sivers article when this email made the rounds earlier this month:

I feel that this story is important to tell you because Kickstarter.com copied us. I tried for 4 years to get people to take Fundable seriously, traveling across the country, even giving a presentation to FBFund, Facebook's fund to stimulate development of new apps. It was a series of rejections for 4 years. I really felt that I presented myself professionally in every business situation and I dressed appropriately and practiced my presentations. That was not
enough. The idiots wanted us to show them charts with massive profits and widespread public acceptance so that they didn't have to take any risks.

All it took was 5 super-connected people at Kickstarter (especially Andy Baio) to take a concept we worked hard to refine, tweak it with Amazon Payments, and then take credit. You could say that that's capitalism, but I still think you should acknowledge people that you take inspiration from. I do. I owe the concept of Fundable to many things, including living in cooperative student housing and studying Political Science at Michigan. Rational choice theory, tragedy of the commons, and collective action are a few political science concepts that are relevant to Fundable.

Yes, Fundable had some technical and customer service problems. That's because we had no money to revise it. I had plans to scrap the entire CMS and start from scratch with a new design. We were just so burned out that motivation was hard to come by. What was the point if we weren't making enough money to live on after 4 years?

The disconnect between idea and execution here is so vast it's hard to understand why the author himself can't see it.

I wouldn't call ideas worthless, per se, but it's clear that ideas alone are a hollow sort of currency. Success is rarely determined by the quality of your ideas. But it is frequently determined by the quality of your execution. So instead of worrying about whether the Next Big Idea you're all working on is sufficiently brilliant, worry about how well you're executing.

The criticism that all you need is "super-connected people" to be successful was also leveled at Stack Overflow. In an email to me last year, Andy Baio — ironically, the very person being cited in the email — said:

I very much enjoyed the Hacker News conversation about cloning the site in a weekend. My favorite comments were from the people that believe Stack Overflow is only successful because of the Cult of Atwood & Spolsky. Amazing.

I don't care how internet famous you are; nobody gets a pass on execution. Sure, you may have a few more eyeballs at the beginning, but if you don't build something useful, the world will eventually just shrug its collective shoulders and move along to more useful things.

One of my all time favorite software quotes is from Wil Shipley:

This is all your app is: a collection of tiny details.

In software development, execution is staying on top of all the tiny details that make up your app. If you're not constantly obsessing over every aspect of your application, relentlessly polishing and improving every little part of it — no matter how trivial — you're not executing. At least, not well.

And unless you work alone, which is a rarity these days, your ability to stay on top of the collection of tiny details that makes up your app will hinge entirely on whether or not you can build a great team. They are the building block of any successful endeavor. This talk by Ed Catmull is almost exclusively focused on how Pixar learned, through trial and error, to build teams that can execute.

It's a fascinating talk, full of some great insights, and you should watch the whole thing. In it, Mr. Catmull amplifies Mr. Sivers' sentiment:

If you give a good idea to a mediocre group, they'll screw it up. If you give a mediocre idea to a good group, they'll fix it. Or they'll throw it away and come up with something else.

Execution isn't merely a multiplier. It's far more powerful. How your team executes has the power to transform your idea from gold into lead, or from lead into gold. That's why, when building Stack Overflow, I was so fortunate to not only work with Joel Spolsky, but also to cherry-pick two of the best developers I had ever worked with in my previous jobs and drag them along with me. Kicking and screaming if necessary.

If I had to point to the one thing that made our project successful, it was not the idea behind it, our internet fame, the tools we chose, or the funding we had (precious little, for the record).

It was our team.

The value of my advice is debatable. But you would do well to heed the advice of Mr. Sivers and Mr. Catmull. If you want to be successful, stop worrying about the great ideas, and concentrate on cultivating great teams.

Go to Source

Opening the file in a different, smarter text editor results in the file displayed properly in multiple paragraphs.

The answer to this puzzle lies in our old friend, invisible characters that we can’t see but that are totally not out to get us. Well, except when they are.

The invisible problem characters in this case are newlines.

Did you ever wonder what was at the end of your lines? As a programmer, I knew there were end of line characters, but I honestly never thought much about them. They just … worked. But newlines aren’t a universally accepted standard; they are different depending who you ask, and what platform they happen to be computing on:

The Carriage Return (CR) and Line Feed (LF) terms derive from manual typewriters, and old printers based on typewriter-like mechanisms (typically referred to as “Daisywheel” printers).

On a typewriter, pressing Line Feed causes the carriage roller to push up one line — without changing the position of the carriage itself — while the Carriage Return lever slides the carriage back to the beginning of the line. In all honesty, I’m not quite old enough to have used electric typewriters, so I have a dim recollection, at best, of the entire process. The distinction between CR and LF does seem kind of pointless — why would you want to move to the beginning of a line without also advancing to the next line? This is another analog artifact, as Wikipedia explains:

On printers, teletypes, and computer terminals that were not capable of displaying graphics, the carriage return was used without moving to the next line to allow characters to be placed on top of existing characters to produce character graphics, underlines, and crossed out text.

So far we’ve got:

• Confusing terms based on archaic hardware that is no longer in use, and is confounding to new users who have no point of reference for said terms;
• Completely arbitrary platform “standards” for what is exactly the same function.

Pretty much business as usual in computing. If you’re curious, as I was, about the historical basis for these decisions, Wikipedia delivers all the newline trivia you could possibly want, and more:

The sequence CR+LF was in common use on many early computer systems that had adopted teletype machines, typically an ASR33, as a console device, because this sequence was required to position those printers at the start of a new line. On these systems, text was often routinely composed to be compatible with these printers, since the concept of device drivers hiding such hardware details from the application was not yet well developed; applications had to talk directly to the teletype machine and follow its conventions. The separation of the two functions concealed the fact that the print head could not return from the far right to the beginning of the next line in one-character time. That is why the sequence was always sent with the CR first. In fact, it was often necessary to send extra characters (extraneous CRs or NULs, which are ignored) to give the print head time to move to the left margin. Even after teletypes were replaced by computer terminals with higher baud rates, many operating systems still supported automatic sending of these fill characters, for compatibility with cheaper terminals that required multiple character times to scroll the display.

CP/M’s use of CR+LF made sense for using computer terminals via serial lines. MS-DOS adopted CP/M’s CR+LF, and this convention was inherited by Windows.

This exciting difference in how newlines work means you can expect to see one of three (or more, as we’ll find out later) newline characters in those “simple” ASCII text files.

If you’re fortunate, you’ll pick a fairly intelligent editor that can detect and properly display the line endings of whatever text files you open. If you’re less fortunate, you’ll see onegiantunbrokenline, or a bunch of extra ^M characters in the file.

Even worse, it’s possible to mix all three of these line endings in the same file. Innocently copy and paste a comment or code snippet from a file with a different set of line endings, then save it. Bam, you’ve got a file with multiple line endings. That you can’t see. I’ve accidentally done it myself. (Note that this depends on your choice of text editor; some will auto-normalize line endings to match the current file’s settings upon paste.)

This is complicated by the fact that some editors, even editors that should know better, like Visual Studio, have no mode that shows end of line markers. That’s why, when attempting to open a file that has multiple line endings, Visual Studio will politely ask you if it can normalize the file to one set of line endings.

This Visual Studio dialog presents the following five (!) possible set of line endings for the file:

1. Windows (CR LF)
2. Macintosh (CR)
3. Unix (LF)
4. Unicode Line Separator (LS)
5. Unicode Paragraph Separator (PS)

The last two are new to me. I’m not sure under what circumstances you would want those Unicode newline markers.

Even if you rule out unicode and stick to old-school ASCII, like most Facebook relationships … it’s complicated. I find it fascinating that the mundane ASCII newline has so much ancient computing lore behind it, and that it still regularly bites us in unexpected places.

If you work with text files in any capacity — and what programmer doesn’t — you should know that not all newlines are created equally. The Great Newline Schism is something you need to be aware of. Make sure your tools can show you not just those pesky invisible white space characters, but line endings as well.

Go to Source

The end result, to my mind, is a device that occupies an uncomfortable, middle ground between laptops and smartphones that tries to please everyone and pleases no one. Consider the factors:

• Size: A bit too large to go into your pocket; a bit too small for regular day-to-day work.
• Power: Slightly more capable than a smartphone; slightly less capable than a laptop.
• Price: Slightly higher than a higher-end smartphone but lacking a phone’s capability and portability; slightly lower than a lower-end notebook but lacking a notebook’s speed and storage.

To summarize: Slightly bigger and pricier than a phone, but can’t phone. Slightly smaller and cheaper than a laptop, but not that much smaller or cheaper. To adapt a phrase I used in an article I wrote yesterday, netbooks are like laptops, but lamer.

This is so wrongheaded I am not sure where to begin. I happen to agree with Dave Winer’s definition of “netbook”:

1. Small size.
2. Low price.
3. Battery life of 4+ hours. Battery can be replaced by user.
4. Rugged.
5. Built-in wifi, 3 USB ports, SD card reader.
6. Runs my software.
7. Runs any software I want; no platform vendor to decide what’s appropriate.
8. Competition. Users have choice and can switch vendors at any time.

Netbooks are the endpoint of four decades of computing — the final, ubiquitous manifestation of “A PC on every desk and in every home”. But netbooks are more than just PCs. If the internet is the ultimate force of democratization in the world, then netbooks are the instrument by which that democracy will be achieved.

No monthly fees and contracts.

No gatekeepers.

Nobody telling you what you can and can’t do with your hardware, or on their network.

To dismiss netbooks as like laptops, but lamer is to completely miss the importance of this pivotal moment in computing — when pervasive internet and the mass production of inexpensive portable computers finally intersected. I’m talking about unlimited access to the complete sum of human knowledge, and free, unfettered communication with anyone on earth. For everyone.

It’s true that smartphones are slowly becoming little PCs, but they will never be free PCs. They will forever be locked behind an imposing series of gatekeepers and toll roads and walled gardens. Anyone with a $199 netbook and access to the internet can make free Skype videophone calls to anywhere on Earth, for as long as they want. Meanwhile, sending a single text message on a smartphone costs 4 times as much as transmitting data to the Hubble space telescope.

I don’t care how “smart” your smartphone is, it will never escape those corporate shackles. Smartphones are simply not free enough to deliver the type of democratic transformation that netbooks — mobile PCs cheap enough and fast enough and good enough for everyone to afford — absolutely will.

That’s why I love netbooks. In all their cheap, crappy glory. And you should too. Because they’re instruments of user power.

The truly significant thing is this — the users took over.

Let me say that again: The users took over.

I always say this is the lesson of the tech industry, but the people in the tech industry never believe it, but this is the loop. In the late 70s and early 80s the minicomputer and mainframe guys said the same kinds of things about Apple IIs and IBM PCs that Michael Dell is saying about netbooks. It happens over and over again, I’ve recited the loops so many times that every reader of this column can recite them from memory. All that has to be said is that it happened again.

Once out, the genie never goes back in the bottle.

Netbooks aren’t an alternative to notebook computers. They are the new computers.

Cheap and crappy? Maybe those early models were, but having purchased a new netbook for $439 shipped, it is difficult for me to imagine the average user ever paying more than $500 for a laptop.

For the price, this is an astonishingly capable PC:

• Dual Core 1.2 GHz Intel CULV Celeron processor
• 2 GB RAM
• Windows 7 Home Premium
• 11.6″ screen with 1366 x 768 resolution
• Thin (1″) and light (3.5 lbs)
• Good battery life (5 hours)
• 3 USB ports, WiFi, webcam, gigabit ethernet

Windows 7 is a fine OS, but this machine would surely be cheaper without the Microsoft Tax, too.

The Acer Aspire 1410 isn’t just an adequate netbook, it’s a damn good computer. At these specifications, it is a huge step up from those early netbook models in every way. But don’t take my word for it; read the reviews at netbooked and Liliputing. (Caveat emptor — there are lots of 1410 models, and the newer dual core CPU version is the one you want.)

Of particular note is the CPU. While the Intel Atom is a technological coup, I don’t feel current Atom CPUs deliver quite enough performance for a modern, JavaScript-heavy, video-intensive internet experience. It is quite clear that Intel is intentionally hobbling newer iterations of the Atom CPU in the name of market segregation, and to prevent too much netbook price erosion.

That’s why the current Intel CULV CPUs are far more attractive options — they’re dramatically faster, and have become power-efficient marvels. I hooked up my watt meter to this Aspire 1410 and I was surprised to find it consume between 13 and 16 watts of power in typical use — while my wife was browsing the web in Firefox, over a wireless connection, with multiple tabs open. I fired up Prime95 torture test to force the CPU to 100% load, and measured 21 watts with one CPU core fully loaded, and 26 watts when both were. These are wall measurements which reflect power conversion inefficiencies of at least 20%, so real consumption was between 10 and 20 watts. I was wondering why it ran so cool; now I know. It barely uses enough power to generate any heat!

Modern netbooks are not cheap and crappy. They’re remarkable computers in their own right, and they’re getting better every day. Which makes me wonder:

A recurring question among Apple watchers for decades has been, “When is Apple going to introduce a low-cost computer?

Steve Jobs answered that decades-old complaint by stating, “We don’t know how to build a sub-$500 computer that is not a piece of junk.”

They may be pieces of junk to Mr. Jobs, but to me, these modest little boxes are marvels — inspiring evidence of the inexorable march of powerful, open computing technology to everyman and everywhere.

We have produced a democracy of netbooks. And the geek in me can’t wait to see what happens next.

Go to Source

Markdown is intended to be as easy-to-read and easy-to-write as is feasible.

Readability, however, is emphasized above all else. A Markdown-formatted document should be publishable as-is, as plain text, without looking like it’s been marked up with tags or formatting instructions. While Markdown’s syntax has been influenced by several existing text-to-HTML filters — including Setext, atx, Textile, reStructuredText, Grutatext, and EtText — the single biggest source of inspiration for Markdown’s syntax is the format of plain text email.

If you’re an ASCII-head of any kind, you will feel immediately at home in Markdown. It was so obviously designed by someone who has done a lot of writing online, as it apes common plaintext conventions that we’ve collectively been using for decades now. It’s certainly far more intuitive than the alternatives I’ve researched.

With a year and a half of real world Markdown experience under our belts on Stack Overflow, we’ve been quite happy. I’d say that Markdown is the worst form of markup except for all the other forms of markup that I’ve tried. Of course, tastes vary, and there are plenty of viable alternatives, but I’d promote Markdown without hesitation as one of the best — if not the best — humane markup options out there.

Not that Markdown is perfect, mind you. After exposing it to a large audience, both Stack Overflow and GitHub independently discovered that Markdown had three particular characteristics that confused a lot of users:

1. URLs are never hyperlinked without placing them in some kind of explicit markup.
2. The underscore [_] can be used to delimit bold and italic, but also works for intra-word emphasis. While a typical use like “_italic_” is clear, there are disturbing and unexpected side effects in phrases such as “some_file_name” and “file_one and file_two”.
3. It is paragraph and not line oriented. Returns are not automatically converted to linebreaks. Instead, paragraphs are detected as one or more consecutive lines of text, separated by one or more blank lines.

Items #1 and #2 are so fundamental and universal that I think they deserve to be changed in the Markdown specification itself. There was so much confusion around unexpected intra-word emphasis and the failure to auto-hyperlink URLs that we changed these Markdown rules before we even came out of private beta. Item #3, the conversion of returns to linebreaks, is somewhat more debatable. I’m on the fence on that one, but I do believe it’s significant enough to warrant an explicit choice either way. It should be a standard configurable option in every Markdown implementation that you can switch on or off depending on the intended audience.

Markdown was originally introduced in 2004, and since then it has gained quite a bit of traction on the web. I mean, it’s no MediaWiki (thank God), but it’s in active use on a bunch of websites, some of them quite popular. And for such a popular form of markup, it’s a bit odd that the last official update to the specification and reference implementation was in late 2004.

Which leads me to the biggest problem with Markdown: John Gruber.

I don’t mean this as a personal criticism. John’s a fantastic writer and Markdown has a (mostly) solid specification, with a strong vision statement. But the fact that there has been no improvement whatsoever to the specification or reference implementation for five years is … kind of a problem.

There are some fairly severe bugs in that now-ancient 2004 Markdown 1.0.1 Perl implementation. Bugs that John has already fixed in eight 1.0.2 betas that have somehow never seen the light of day. Sure, if you know the right Google incantations you can dig up the unreleased 1.0.2b8 archive, surreptitiously posted May 2007, and start prying out the bugfixes by hand. That’s what I’ve had to do to fix bugs in our open sourced C# Markdown implementation, which was naturally based on that fateful (and technically only) 1.0.1 release.

I’d also expect a reference implementation to come with some basic test suites or sample input/output files so I can tell if I’ve implemented it correctly. No such luck; the official archives from Gruber’s site include the naked Perl file along with a readme and license. The word “test” does not appear in either. I had to do a ton more searching to finally dig up MDTest 1.1. I can’t quite tell where the tests came from, but they seem to be maintained by Michel Fortin, the author of the primary PHP Markdown implementation.

But John Gruber created Markdown. He came up with the concept and the initial implementation. He is, in every sense of the word, the parent of Markdown. It’s his baby.

As Markdown’s “parent”, John has a few key responsibilities in shepherding his baby to maturity. Namely, to lead. To set direction. Beyond that initial 2004 push, he’s done precious little of either. John is running this particular open source project the way Steve Jobs runs Apple — by sheer force of individual ego. And that sucks.

Since then, all I can find is sporadic activity on obscure mailing lists and a bit of passive-aggressive interaction with the community.

On 15 Mar 2008, at 02:55, John Gruber wrote:

I despise what you’ve done with Text::Markdown, which is to more or less make it an alias for MultiMarkdown, almost every part of which I disagree with in terms of syntax additions.

Wow, that’s pretty strong language. I’m glad I’m provoking strong opinions, and it’s nice to see you actively contributing to Markdown’s direction

Personally, I don’t actually like (or use) the MultiMarkdown extensions. As noted several times on list, I do not consider what I’ve done to in any way be a good solution technically / internally in it’s current form, and as such
Markdown.pl is still a better ‘reference’ implementation.

However I find it somewhat ironic that you can criticise an active effort to actually move Markdown forwards (who’s current flaws have been publicly acknowledged), when it passes more of your test suite than your effort does, and when you haven’t even been bothered to update your own website about the project since 2004, despite having updated the code which can be found on your site (if you dig) much more recently than this.

I despise copy-pasted code, and forks for no (real) reason – seeing another two dead copies of the same code on CPAN made me sad, and so I’ve done something to take the situation forwards. Maybe if you’d put the effort into maintaining a community and taking Markdown.pl forwards at any time within the last 4 years, you wouldn’t be in a situation where people have taken ‘your baby’ and perverted it to a point that you despise. If starting with Markdown.pl and going forwards with that had been an option, then that would have been my preferred route – but I didn’t see any value in producing what would have been a fifth perl Markdown implementation.

It’s almost at the point where John Gruber, the very person who brought us Markdown, is the biggest obstacle preventing Markdown from moving forward and maturing. It saddens me greatly to see such negligent open source code parenting. Why work against the community when you can work with it? It doesn’t have to be this way. And it shouldn’t be.

I think the most fundamental problem with Markdown, in retrospect, is that the official home of the project is a static set of web pages on John’s site. Gruber could have hosted the Markdown project in a way that’s more amenable to open source collaboration than a bunch of static HTML. I’m pretty sure SourceForge was around in late 2004, and there are lots of options for proper open source project hosting today — GitHub, Google Code, CodePlex, and so forth. What’s stopping him from setting up shop on any of those sites with Markdown, right now, today? Markdown is Gruber’s baby, without a doubt, but it’s also bigger than any one person. It’s open source. It belongs to the community, too.

Right now we have the worst of both worlds. Lack of leadership from the top, and a bunch of fragmented, poorly coordinated community efforts to advance Markdown, none of which are officially canon. This isn’t merely incovenient for anyone trying to find accurate information about Markdown; it’s actually harming the project’s future. Maybe it’s true that you can’t kill an open source project, but bad parenting is surely enough to cause it to grow up stunted and maybe even a little maladjusted.

I mean no disrespect. I wouldn’t bring this up if I didn’t care, if I didn’t think the project and John Gruber were both eminently worthy. Markdown is a small but important part of the open source fabric of the web, and the project deserves better stewardship. While the community can do a lot with the (many) open source orphan code babies out there, they have a much, much brighter future when their parents take responsibility for them.

Go to Source

• Building a PC, Part I: Minimal boot
• Building a PC, Part II: Burn in
• Building a PC, Part III: Overclocking
• Building a PC, Part IV: Now It’s Your Turn
• Building a PC, Part V: Upgrading

Now, lest you think I am some kind of freakish, cave-dwelling luddite, what with my ancient two and a half year old PC, I have upgraded the CPU, upgraded the hard drive, and upgraded the video card since then. I also went from 4 GB of RAM to 8 GB of RAM, but I didn’t happen to blog about that. Normal computers age in dog years — every year they get seven years older — but mine isn’t so bad with all my upgrades! I swear!

Judge for yourself; here’s a picture of it.

But seriously.

A big part of the value proposition of building your own PC is upgrading it in pieces and parts over time. When you’re unafraid to pop the cover off and get your hands dirty with a little upgrading, you can spend a lot less to stay near the top of the performance heap over time. It’s like the argument for buying a car versus renting it; the smart buyers keep the car for as long as possible to maximize the value of their investment. That’s what we’re doing here with our upgrades, and a rebuild is the ultimate upgrade.

In defense of my creaky old computer, the Core 2 series from Intel has been unusually strong over time, one of their best overall platforms in recent memory. It was almost good enough to banish the excerable Pentium 4 series from my mind. Man those were horrible! But the Core 2 series was a solid design with some serious legs; it and scaled brilliantly, from single to dual to quad core, and in frequency from 1 GHz to 3.5 GHz.

I was initially unimpressed with the new Core i7 architecture that Intel launched to replace the Core 2. While the new Nehalem architecture is a huge win on servers, it’s kind of “meh” on the desktop. I have endless battles with overzealous developers who swear up and down that they use their desktops like servers. Sure you do! And you’re building the space shuttle with it, right? Of course you are. Yeah.

Meanwhile, back on Planet Desktop, there were some other reasons that I started thinking seriously about upgrading from my overclocked Core 2 Duo to a Core i7 upgrade:

• The Core i7 platform uses triple channel DDR3 memory. While the benefits of the additional bandwidth are somewhat debatable on the desktop (as usual), one interesting side-effect is that motherboards have 6 memory slots. While 16 GB is theoretically possible on Core 2 systems, it required extremely expensive 8 GB DIMMs. But with 6 memory slots, we can achieve 12 GB without breaking the bank — by using six 2 GB DIMMs.
• The Core i7 is Intel’s first “real” quad-core architecture. Intel’s previous quad core CPUs were basically two dual core CPUs duct taped together on the same die. No such shortcuts were taken with the i7. While the difference is sort of academic, there are some smallish real world performance implications.
• Mainstream software is finally ready for quad core CPUs. It’s not uncommon today to find applications and games that can actually use two CPU cores reasonably effectively, and those that can use four or more cores are not the extreme rarity they used to be. Don’t get me wrong, scaling well to four or more CPU cores is still rare, but it’s no longer spit-take rare.
• Intel introduced the mainstream second generation Core i5 series, so the platform is fairly mature. All the new architecture bugs are worked out. It’s also less prohibitively expensive than it was when it was when it was introduced.

At this point, I had the seven year upgrade itch really bad. My 3.8 GHz Core 2 Duo with 8 GB of RAM was not exactly chopped liver, but I started fantasizing a lot about the prospect of having a next generation quad-core CPU (of similar clock speed) with hyperthreading and 12 GB of RAM.

If you’re wondering why I need this, or why in fact anyone would need such an embarrassment of desktop power, then I’ll refer you to my friend Loyd Case.

Don’t ask me why I need six cores and 24GB. To paraphrase a Zen master, if you have to ask, you do not know.

Loyd has indirectly brought up another reason to choose the i7 platform; it’s pin-compatible with Intel’s upcoming “Gulftown” high end 6-core CPU. So, your upgrade path is clear. (It’s also rumored that the next iteration of the Mac Pro will have two of these brand new 6-core CPUs, before any other vendor gets access to them, which is totally plausible.)

As far as I’m concerned, until everything on my computer happens instantaneously, my computer is not nearly fast enough. Besides, relative to how much my time costs, these little $200-$500 upgrades to get amazing performance are freakin’ chump change. If I save a measly 15 minutes a day, it’s worth it. As I like to remind pointy-haired managers all over the world, Hardware is Cheap, and Programmers are Expensive. OK, maybe I’m biased, but the conclusion was overwhemingly clear: it’s UPGRAYEDD time!

This is a more than an upgrade, though, it’s a rebuild — a platform upgrade. That means I’ll be assembling the following …

• new Motherboard
• new RAM
• new CPU
• new heatsink

… and dropping that into my existing system, which is highly optimized for silence. The case, power supply, hard drives, DVD-R, etc won’t change. On the outside, it’ll look the same, but on the inside, it’s a whole new PC. This is analogous to replacing the engine in a sports car, I suppose. On the outside, it will appear to be the same car, but there’s a lot more horses under the hood.

As I said in the first part of my building your own PC series, if you can assemble a LEGO kit, you can build a PC.

Take your time, be careful, and go in the right order. So, first things first. Let’s assemble the CPU, heatsink, and memory on the motherboard — in that specific sequence, because modern heatsinks can be a pain to attach.

Man, check out at all that hot, sweet, PC hardware! I get a little residual thrill just cropping the picture. Love this stuff! Anyway, that gives us a mountable motherboard with all the important bits pre-installed:

• ASRock X58 Extreme motherboard ($169)
  Inexpensive, has all the essential features I care about, and is recommended by Tom’s Hardware. I’m not into fancy, spendy motherboards; I think they’re a ridiculous waste of money.

• XIGMATEK HDT-S1283 cooler ($35).
  Direct contact between the CPU cooler heatpipes and the CPU surface is the new hotness, or rather, coolness. It really works, since all the top performing CPU coolers use it now. This one is fairly inexpensive at $35 and gets great reviews. Also, I highly recommend the optional screw mount kit ($8). Modern CPU coolers are large, and the mounting mechanism needs to be more solid than plastic pushpins.

• Kingston HyperX 4GB (2 x 2GB) DDR3 2000 ($135) × 3
  I’ve had good luck with Kingston in the past. I went with their semi-premium brand this time, as I plan to do a bit of overclocking and the price difference is fairly small. Remember, this is a 12 GB build, so we’ll need three of these kits to populate all 6 memory slots on the motherboard.

• Intel Core i7-960 3.2 GHz CPU ($590)
  While you could make a very solid argument that the Core i7-920 CPU ($289) is a better choice because it’s identical and overclocks to the same level, I was willing to spend a bit more here as “insurance” that I get to the magical 3.8 Ghz level that my old Core 2 Duo was overclocked to.

It adds up to about $1000 all told. A rebuild is definitely more expensive than one-off upgrades of CPU, memory, and hard drive. But, remember, this is a rebuild of my PC — and a fire-breathing, top of the line performance rebuild at that. That takes spending a moderate (but not exorbitant) amount of money.

Now that we’ve got all that stuff assembled, the next thing to do is open my existing PC, disconnect all the cables going to the motherboard, temporarily remove any expansion cards, unscrew the motherboard and lift it out.

Once the old motherboard assembly was pulled out, I plopped in the new motherboard, screwed it down, and reattached the cables and expansion cards. Don’t close up the PC at this point, though. Before powering it on, double check and make sure all the cables are reattached correctly:

• Power cables from the PSU to the motherboard. There are usually at least two, on modern PCs.
• Hard drive cables from the HDDs to the motherboard.
• Power switch, Reset switch, Activity light cables. Without the power switch connected, good luck powering up. This motherboard happens to have built-in power and reset switches for testing, but most don’t.
• Fan connectors from the Heatsink and case fans to the motherboard.
• Power cables from the PSU to the video card, if you have a fancy video card.

If anything is wrong, we’ll just have to re-open the case again. On top of that, we need to monitor temperatures and airflow, and that’s much easier with the case open.

Fortunately, my rebuild booted up on the first try. If you’re not so lucky, don’t fret! Disconnect the power cord, then go back and re-check everything. I get it wrong, sometimes, too; I actually forgot to reconnect the video card power connectors, and was wondering why only the secondary video card was booting up. Once I re-checked, I immediately saw my mistake, fixed it, and rebooted.

Once you have a successful boot, don’t even think about booting into the operating system yet. Enter the BIOS (this is typically done by pressing F12 or Delete during bootup) and check the BIOS screens to make sure it’s detecting your hard drives, memory, and any optical drives successfully. Browse around and do some basic reality checks. Then do not pass GO, do not collect $200, go straight to your motherboard manufacturer’s website and download the latest BIOS. On another computer, obviously. Most modern motherboards allow updating the BIOS from a USB key, so just copy the BIOS files on the USB key, reboot, and use the BIOS menus to update. After you’ve updated the BIOS, set BIOS options to taste, and we’re finally ready to boot into an operating system.

While this may sound like a lot of work, it really isn’t. All told it was maybe an hour, tops. I’m fairly experienced at this stuff, but it’s fundamentally not that complicated; it’s still just a very fancy adult LEGO kit.

Courtesy of this $1000 rebuild, my ancient 2.5 year old PC is reborn as a completely new state-of-the-art PC, at least internally. That was always part of the plan! Next up — once we’ve proven that it’s stable in typical use — overclocking, naturally. I’ll have more on that in a future blog post, but I can tell you right now that Core i7 overclocking is … interesting.
Go to Source

Here’s what happened:

1. The server experienced routine hard drive failure.
2. Because of the hard drive failure, the virtual machine image hosting this blog was corrupted.
3. Because the blog was hosted in a virtual machine, the standard daily backup procedures at the host were unable to ever back it up.
4. Because I am an idiot, I didn’t have my own (recent) backups of Coding Horror. Man, I wish I had read some good blog entries on backup strategies!
5. Because there were no good backups, there was catastrophic data loss. Fin, draw curtain, exeunt stage left.

At first, I was upset with our provider, CrystalTech.

I am still confused how the most common, routine, predictable, and mundane of server hardware failures — losing a mechanical hard drive — could cause such extreme data loss carnage. What about, oh, I don’t know, a RAID array? Aren’t they designed to prevent this kind of single point of failure drive loss catastrophe? Isn’t a multi drive RAID array sort of standard on datacenter servers? I know we have multi-drive RAID arrays are on all of our Stack Overflow servers.

I also wish their routine backup procedures had greater awareness of virtual machine images. While I’ll grant you that backing up a live virtual machine is somewhat complex, and typically requires special operating system support and API hooks, it is not exactly an unknown science at this point in time. Heck, At the very least, just let us know when the backup has been regularly failing each day, every day, for years.

Then I belatedly realized that this was, after all, my data. And it is irresponsible of me to leave the fate of my data entirely in someone else’s hands, regardless of how reliable they may or may not be. Responsibility for my data begins with me. If I haven’t taken appropriate measures, who am I to cast aspersions on others for not doing the same? Glass houses and all that.

So, I absolve CrystalTech of all responsibility in this matter. They’ve given us a great deal on our dedicated server, and performance and reliability (with one recent, uh… exception) have been excellent to date. It is completely my fault that I neglected to have proper backups in place for Coding Horror. Well, technically, I did have a backup but it was on the virtual machine itself. Does that count? No? Halfsies?

Apparently, I was gambling that nothing bad would ever happen at the datacenter. Because that’s what you’re doing when you run without your own backups. Gambling.

I’ll add gambling to the long, long list of things I suck at.

Now that I’ve apologized, it’s time to let the healing begin. And by healing, I mean the excruciatingly painful process of reconstructing Coding Horror from internet caches and the few meager offsite backups I do have. My first order of business was to ask on SuperUser what strategies people recommend for recovering a lost website with no backup. Strategies other than berating me for my obvious mistake. Also, comments are currently disabled while the site is being reconstructed from static HTML. Oh, darn!

I’ll let my son Rock Hard Awesome stand in for the zinger of a comment that I know some of you were just dying to leave.

I’m not saying I don’t deserve it. Consider me totally zingatized.

I mentioned my woes on Twitter and I was humbled by the outpouring of community support. Thanks to everyone who reached out with support of any kind. It is greatly appreciated.

I was able to get a static HTML version of Coding Horror up almost immediately thanks to Rich Skrenta of blekko.com. He kindly provided a tarball of every spidered page on the site. Some people have goals, and some people have big hairy audacious goals. Rich’s is especially awe-inspiring: taking on Google on their home turf of search. That’s why he just happened to have a complete text archive of Coding Horror at hand. Rich, have I ever told you that you’re my hero? Anyway, you’re viewing the static HTML version of Coding Horror right now thanks to Rich. Surprisingly, there’s not a tremendous amount of difference between a static HTML version of this site and the live site. One of the benefits of being a minimalist, I suppose.

That pretty much solved all my text post recovery problems in one fell swoop. Through this process, I’ve learned that anything even remotely popular you put on the web will be archived as text, forever, by a dozen different web spiders. I don’t think you can actually lose text you post on the web. Not in any meaningful sense; I’m not sure it’s possible. As long as you’re willing to spend the time digging through web spider archives in some form (and yes, I did cheat mightily), you can always get textual content back, all of it.

The blog images, however, are another matter entirely. I have learned the hard way that there are almost no organizations spidering and storing images on the web. Yes, there is archive.org, and God bless ‘em for that. But they have an impossible job they’re trying to do with limited resources. Beyond that, there’s … well, frankly, a whole lot of nothing. A desperate, depressing void of nothing. In fact, if you can only back up one thing on your public website, it should be the images. Because that’s the thing you’ll have the most difficulty recovering when catastrophe happens. I’m planning to donate $100 to archive.org, as I have a whole new appreciation for how rare an internet-wide full archive service — one that includes images — really is.

That said, There are some limited, painful avenues to explore for recovering lost website images. I started with an ancient complete backup from mid 2006 with full images. And then Maciej Ceglowski of the nifty full-archive bookmarking service pinboard.in generously contributed about 200 blog posts that he had images for.

I also went through a period when I was going on a bandwidth diet and experimenting with hosting Coding Horror images elsewhere on the web. I’m slowly going through and recovering images locally from there. Beyond that, several avid Coding Horror readers contributed some archived images — so thanks to Yasushi Aoki, Marcin Gołębiowski, Peter Mortensen, and anybody else I’ve forgotten.

Also, I should point out that a few enterprising programmers have proposed clever schemes for automatic recovery of images, such as Niyaz with his blog post Get cached images from your visitors, and John Siracusa with his highly voted 304 idea. I haven’t had time to follow up on these yet but they seem plausible to me.

I’ve restored all the images I have so far, but it’s still woefully incomplete. The most important part of Coding Horror is definitely the text of the posts, but I do have some regrets that I’ve lost key images from many blog posts, including those about my son. It feels like irresponsible parenting, in the broadest possible sense of the words.

The process of image recovery is still ongoing. If you’d like to contribute lost Coding Horror images, please do. I’d be more than happy to mail stickers on my dime to anyone who contributes an image that is currently a 404 on the site.

What can we all learn from this sad turn of events?

1. I suck.
2. No, really, I suck.
3. Don’t rely on your host or anyone else to back up your important data. Do it yourself. If you aren’t personally responsible for your own backups, they are effectively not happening.
4. If something really bad happens to your data, how would you recover? What’s the process? What are the hard parts of recovery? I think in the back of my mind I had false confidence about Coding Horror recovery scenarios because I kept thinking of it as mostly text. Of course, the text turned out to be the easiest part. The images, which I had thought of as a “nice to have”, were more essential than I realized and far more difficult to recover. Some argue that we shouldn’t be talking about “backups”, but recovery.
5. It’s worth revisiting your recovery process periodically to make sure it’s still alive, kicking, and fully functional.
6. I’m awesome! No, just kidding. I suck.

So when, exactly, is International Backup Awareness Day? Today. Yesterday. This week. This month. This year. It’s a trick question. Every day is International Backup Awareness Day. And the sooner I figure that out, the better off I’ll be.

Go to Source

Designed for humans first and machines second, microformats are a set of simple, open data formats built upon existing and widely adopted standards.

The official microformat “elevator pitch” tells us nothing useful. That’s not a good sign. It doesn’t get much better on the learn more link, either.

I’m left scratching my head, wondering why I should care. What problem, exactly, do microformats solve for me as a user? As a software developer? There’s lots of hand-wavy talk about data, but precious little in the way of concrete stories or real world examples.

But I have a real world example: a CV. To some human resource departments the standard web interchange format for a CV or Resume is already established — it’s called “Microsoft Word”. I have no beef with Word, but certainly we’d like to pick a more simple, open data format for our personal data than Microsoft Word — and the hResume microformat seems to fit the bill. And if your CV is published on the web in a standard(ish) format, it’s easier to take it with you wherever you need to go.

I had already implemented the tag and identity microformats on Stack Overflow many months ago. I wasn’t convinced of the benefits, but the implementation was so easy that it seemed like more work to argue the point than to actually get it done. Judge for yourself:

<a href="http://www.codinghorror.com/" rel="me">codinghorror.com</a>
<a href="/questions/tagged/captcha" rel="tag">captcha</a>

Fairly clean and simple, right? That was the extent of my experience with microformats. Limited, but positive. Then I read through the hResume microformat spec. You should read it too. Go ahead. I’ll wait here.

My first impression was not positive, to put it mildly. So you want me to take the ambiguous, crappy “HTML” markup we already have and layer some ambiguous, crappy “microformat” markup on top of it? And that’s … a solution? If that’s what microformats are going to be about, I think I might want off the microbus.

Let’s take a look at a representative slice of hResume markup:

<div class="vcard">
  <a class="fn org url" href="http://example.com/">Example</a>
  <div class="adr">
    <span class="type">Work</span>:
    <div class="street-address">169 Maple Ave</div>
    <span class="locality">Anytown</span>,
    <abbr class="region" title="Iowa">IA</abbr>
    <span class="postal-code">50981</span>
    <div class="country-name">USA</div>
  </div>
</div>

As you can see, the crux of microformats is overloading CSS classes. When you give something the “adr” class within the “vcard” class, that means it’s the address data field within the hCard, within the hResume.

While I can see the attraction, this approach makes me uneasy:

1. We’re overloading the class attribute with two meanings. Is “adr” the name of a class that contains styling information, or the name of a data field? Or both? It’s impossible to tell. The minute you introduce a microformat into your HTML, the semantics of the class attribute have been permanently altered.
2. The microformat css class names may overlap with existing css classes. Woe betide the poor developer who has to retrofit a microformat on an established site where “locality” or “region” have already been defined in the CSS and are associated with elements all over the site. And let me tell you, many of the microformat css field names are, uh, conveniently named what you’ve probably already used in your HTML somewhere. In the wrong way, inevitably.
3. There’s no visual indication whatsoever that any given css class is a microformat. If you hire a new developer, how can they possibly be expected to know that “postal-code” isn’t just an arbitrarily chosen CSS class name, it’s a gosh darned officially blessed microformat? What if they decide they don’t like dashes in CSS class names and rename the style “postalcode”? Wave bye bye to your valid microformat. If it seems fragile and obtuse, that’s because it is.
4. The spec is incredibly ambiguous. I read through the hResume, hCard, and hCalendar spec multiple times, checked all the samples, viewed source on existing sites, used all the validators I could find, and I still got huge swaths of the format wrong! For a “simple” and “easy” format, it’s … anything but, in my experience. The specification is full of ambiguities and requires a lot of interpretation to even get close. I’m not the world’s best developer, but I’m theoretically competent, and if I can’t implement hResume without wanting to cut myself and/or writing snarky blog posts like this, how can we expect everyone else to?
5. It doesn’t handle unstructured data well. On Stack Overflow, we have a single “location” field. No city, state, zip, lat, long, and all that jazz: just an unstructured, freeform, enter-whatever-pleases-you “location” field. This was awkward to map in hCard, which practically demands that addresses be chopped up into meticulous little sub-fields. This is a bit ironic for a format supposedly designed to work with the loose, unstructured world wide web. Oh, and this goes double for dates. If you don’t have an ISO datetime value, good luck.

Maybe I have a particular aversion to getting my chocolate data structure mixed up with my peanut butter layout structure, but it totally skeeves me out that the microformat folks actually want us to design our CSS and HTML around these specific, ambiguous and non-namespaced microformat CSS class names. It feels like a hacky overload. While you could argue this is no different than the web and HTML in general — a giant wobbly teetering tower of nasty, patched-together hacks — something about it fundamentally bothers me.

Now, all that said, I still think microformats are useful and worth implementing, if for no other reason than it’s too easy not to. If you have semi-structured data, and it maps well to an existing microformat, why not? Yes, it is kind of a hack, but it might even be a useful hack if Google starts indexing your microformats and presenting them in search results. While I’m unclear on the general benefits of microformats for end users or developers, seeing stuff like this in search results …

… is enough to convince me that microformats are a step in the right direction. Warts and all. While we’re waiting for HTML5 and its mythical data attributes to ship sometime this century, it’s better than nothing.

Go to Source

• The schedule was too aggressive and too short. We need more time!
• We ran into unforeseen technical problems that forced us to make compromises we are uncomfortable with.
• We had the wrong design, and needed to change it in the middle of development.
• Our team experienced internal friction between team members that we didn’t anticipate.
• The customers weren’t who we thought they were.
• Communication between the designers, developers, and project team wasn’t as efficient as we thought it would be.
• We overestimated how quickly we could learn a new technology.

The list goes on and on. Reasons for failure on a software project are legion.

At the end of the development cycle, you end up with software that is a pale shadow of the shining, glorious monument to software engineering that you envisioned when you started.

It’s tempting, at this point, to throw in the towel — to add more time to the schedule so you can get it right before shipping your software. Because, after all, real developers ship.

I’m here to tell you that this is a mistake.

Yes, you did a ton of things wrong on this project. But you also did a ton of things wrong that you don’t know about yet. And there’s no other way to find out what those things are until you ship this version and get it in front of users and customers. I think Donald Rumsfeld put it best:

As we know,
There are known knowns.
There are things we know we know.
We also know
There are known unknowns.
That is to say
We know there are some things
We do not know.
But there are also unknown unknowns,
The ones we don’t know
We don’t know.

In the face of the inevitable end-of-project blues — rife with compromises and totally unsatisfying quick fixes and partial soutions — you could hunker down and lick your wounds. You could regroup and spend a few extra months fixing up this version before releasing it. You might even feel good about yourself for making the hard call to get the engineering right before unleashing yet another buggy, incomplete chunk of software on the world.

Unfortunately, this is an even bigger mistake than shipping a flawed version.

Instead of spending three months fixing up this version in a sterile, isolated lab, you could be spending that same three month period listening to feedback from real live, honest-to-god, annoyingdedicated users of your software. Not the software as you imagined it, and the users as you imagined them, but as they exist in the real world. You can turn around and use that directed, real world feedback to not only fix all the sucky parts of version 1, but spend your whole development budget more efficiently, predicated on hard usage data from your users.

Now, I’m not saying you should release crap. Believe me, we’re all perfectionists here. But the real world can be a cruel, unforgiving place for us perfectionists. It’s saner to let go and realize that when your software crashes on the rocky shore of the real world, disappointment is inevitable … but fixable! What’s important isn’t so much the initial state of the software — in fact, some say if you aren’t embarrassed by v1.0 you didn’t release it early enough — but what you do after releasing the software.

The velocity and responsiveness of your team to user feedback will set the tone for your software, far more than any single release ever could. That’s what you need to get good at. Not the platonic ideal of shipping mythical, perfect software, but being responsive to your users, to your customers, and demonstrating that through the act of continually improving and refining your software based on their feedback. So to the extent that you’re optimizing for near-perfect software releases, you’re optimizing for the wrong thing.

There’s no question that, for whatever time budget you have, you will end up with better software by releasing as early as practically possible, and then spending the rest of your time iterating rapidly based on real world feedback.

So trust me on this one: even if version 1 sucks, ship it anyway.

Go to Source

But not every programmer does something about the bad code they’re polluting the world with, day in and day out. There’s a whole universe of possibilities:

• Follow the instructions on the paint can
• Become a software apprentice
• Get a coding buddy
• Practice the fundamentals
• A program of effortful study
• Participate in the community to sharpen your saw

But that’s a lot of work. Really freaking hard work! Wouldn’t it be nice if you could do something a bit simpler and easier to, just … say … offset the bad code you’re producing?

Well, now you can — with Bad Code Offsets.

I am a proud member of the Alliance for Code Excellence, and this is our vision:

We envision a world where software runs cleanly and correctly as it simplifies, enhances and enriches our day to day work and home lives. Mitigating the scope and negative impact of bad code on our jobs, our lives and our world is our all-consuming passion. We foresee a time when bad coding practices and their rotten fruits have been eliminated from this earth and its server farms thereby heralding a new age of software brilliance and efficacy.

Nettlesome bugs and poorly written code have been constant impediments towards realizing our full potential as programmers and engineers. Bad Code Offsets provides the vehicle for balancing the scales of poor past practice while freeing us to pursue current excellence in code development. Until the dawn of the worldwide, bug free code base, each of us can take steps towards reducing our bad code footprint and remediate the bad code that we have each individually and collectively left behind on the desktops, servers and mainframes at school, at work and at home.

Yes, this is partly tongue in cheek, but we aren’t just doing it for the lulz. Bad code offsets cost real money, because the Alliance has a goal:

Q: Where does my money go?

A: The proceeds from the sale of Bad Code Offsets are donated to various worthy Open Source initiatives that are carrying the fight against bad code on a daily basis. These organizations include:

• jQuery
• PostgreSQL
• The Apache Software Foundation

This is the awesome part: the money you spend on Bad Code Offsets really does offset bad code!

All the money spent on bad code offsets goes directly to open source projects that actively make programmers’ lives better. For every ten thousand lines of mind-bendingly bad code produced, we hope to subsidize a thousand lines of quality open source code.

So, please — buy bad code offsets today. It is, quite literally, the least you could do.

Go to Source

You can’t parse [X]HTML with regex. Because HTML can’t be parsed by regex. Regex is not a tool that can be used to correctly parse HTML. As I have answered in HTML-and-regex questions here so many times before, the use of regex will not allow you to consume HTML.

Regular expressions are a tool that is insufficiently sophisticated to understand the constructs employed by HTML. HTML is not a regular language and hence cannot be parsed by regular expressions. Regex queries are not equipped to break down HTML into its meaningful parts. so many times but it is not getting to me. Even enhanced irregular regular expressions as used by Perl are not up to the task of parsing HTML. You will never make me crack. HTML is a language of sufficient complexity that it cannot be parsed by regular expressions.

Even Jon Skeet cannot parse HTML using regular expressions. Every time you attempt to parse HTML with regular expressions, the unholy child weeps the blood of virgins, and Russian hackers pwn your webapp. Parsing HTML with regex summons tainted souls into the realm of the living. HTML and regex go together like love, marriage, and ritual infanticide. The <center> cannot hold it is too late. The force of regex and HTML together in the same conceptual space will destroy your mind like so much watery putty. If you parse HTML with regex you are giving in to Them and their blasphemous ways which doom us all to inhuman toil for the One whose Name cannot be expressed in the Basic Multilingual Plane, he comes.

That’s right, if you attempt to parse HTML with regular expressions, you’re succumbing to the temptations of the dark god Cthulhu’s … er … code.

This is all good fun, but the warning here is only partially tongue in cheek, and it is born of a very real frustration.

I have heard this argument before. Usually, I hear it as justification for seeing something like the following code:

 # pull out data between <td> tags
($table_data) = $html =~ /<td>(.*?)<\/td>/gis;

“But, it works!” they say.
“It’s easy!”
“It’s quick!”
“It will do the job just fine!”

I berate them for not being lazy. You need to be lazy as a programmer. Parsing HTML is a solved problem. You do not need to solve it. You just need to be lazy. Be lazy, use CPAN and use HTML::Sanitizer. It will make your coding easier. It will leave your code more maintainable. You won’t have to sit there hand-coding regular expressions. Your code will be more robust. You won’t have to bug fix every time the HTML breaks your crappy regex

For many novice programmers, there’s something unusually seductive about parsing HTML the Cthulhu way instead of, y’know, using a library like a sane person. Which means this discussion gets reopened almost every single day on Stack Overflow. The above post from five years ago could be a discussion from yesterday. I think we can forgive a momentary lapse of reason under the circumstances.

Like I said, this is a well understood phenomenon in most programming circles. However, I was surprised to see a few experienced programmers in metafilter comments actually defend the use of regular expressions to parse HTML. I mean, they’ve heeded the Call of Cthulhu … and liked it.

Many programs will neither need to, nor should, anticipate the entire universe of HTML when parsing. In fact, designing a program to do so may well be a completely wrong-headed approach, if it changes a program from a few-line script to a bullet-proof commercial-grade program which takes orders of magnitude more time to properly code and support. Resource expenditure should always (oops, make that very frequently, I about overgeneralized, too) be considered when creating a programmatic solution.

In addition, hard boundaries need not always be an HTML-oriented limitation. They can be as simple as “work with these sets of web pages”, “work with this data from these web pages”, “work for 98% users 98% of the time”, or even “OMG, we have to make this work in the next hour, do the best you can”.

We live in a world full of newbie PHP developers doing the first thing that pops into their collective heads, with more born every day. What we have here is an ongoing education problem. The real enemy isn’t regular expressions (or, for that matter, goto), but ignorance. The only crime being perpetrated is not knowing what the alternatives are.

So, while I may attempt to parse HTML using regular expressions in certain situations, I go in knowing that:

• It’s generally a bad idea.
• Unless you have discipline and put very strict conditions on what you’re doing, matching HTML with regular expressions rapidly devolves into madness, just how Cthulhu likes it.
• I had what I thought to be good, rational, (semi) defensible reasons for choosing regular expressions in this specific scenario.

It’s considered good form to demand that regular expressions be considered verboten, totally off limits for processing HTML, but I think that’s just as wrongheaded as demanding every trivial HTML processing task be handled by a full-blown parsing engine. It’s more important to understand the tools, and their strengths and weaknesses, than it is to knuckle under to knee-jerk dogmatism.

So, yes, generally speaking, it is a bad idea to use regular expressions when parsing HTML. We should be teaching neophyte developers that, absolutely. Even though it’s an apparently neverending job. But we should also be teaching them the very real difference between parsing HTML and the simple expedience of processing a few strings. And how to tell which is the right approach for the task at hand.

Whatever method you choose — just don’t leave the <cthulhu> tag open, for humanity’s sake.

Go to Source

Also, how exactly is that day is different from any other? But seriously.

Code that is visible is code that can be wrong. No surprise there. But did you know that even the code you can’t see may be wrong, too?

These are the questions that drive young programmers to madness. Take this perfectly innocent code, for example.

Looks fine, doesn’t it? But hold on. Wait a second. Let’s take another, closer look.

OH. MY. GOD!

If you’re not a programmer, you may be looking at these two images and wondering what the big deal is. That’s fine. But I humbly submit that, well, you’re not one of us. You don’t appreciate what it’s like to spend every freaking minute of every freaking day agonizing over the tiniest details of the programs you write. Not because we want to, you understand, but because the world explodes when we don’t.

I mean that literally. Well, almost. If one semicolon is out of place, everything goes sideways. That’s how programming works. It’s fun! Sometimes! I swear!

We got into this industry because, quite frankly, we are control freaks. It’s who we are. It’s what we do. Now to imagine, to our dismay, that there’s all this stupid, useless whitespace at the ends of our lines. Stuff that’s there, but we can’t see it. Well, those are the nightmares OCD horror movies are made of. I have a full-body itchiness just talking about it.

Depending on how far down the rabbit-hole you want to go, there’s any number of things you could do here:

• Have a post-build step, perhaps something with a regular expression like \s*?$ in it, that auto-cleans extra spaces checked into source control
• Execute a local macro which removes whitespace from ends of lines
• Have a special rule to highlight extra spaces
• Run your IDE in whitespace-always-visible mode, or toggle it frequently

OK, fine, so maybe the world won’t explode if there are a few extra bits of whitespace in my code.

But all the same, I think I’ll go back and make extra double plus sure no more of that pesky whitespace has accumulated in my code when I wasn’t looking. Just because I can’t see it doesn’t mean it’s not out to get me.

Go to Source

That’s where Jason Scott comes in.

You may know Jason Scott from BBS The Documentary. Or, perhaps you’re familiar with textfiles.com, his massive (and growing) archive of what passed for blogs and forums in the earliest online era.

A wonderful thing happened in the 1980s: Life started to go online. And as the world continues this trend, everyone finding themselves drawn online should know what happened before, to see where it all really started to come together and to know what went on, before it’s forgotten.

When a historian or reporter tries to capture the feelings and themes that proliferated through the BBS Scene of the early 1980′s, the reader nearly always experiences a mere glimpse of what went on. This is probably true of most any third-party reporting, but when the culture is your own, and when the experiences were your own, the gap between story and reality is that much wider, and it’s that much harder to sit back and let the cliche-filled summary become “The Way It Was.” You want to do something, anything so that the people who stumble onto the part of history that was yours know what it was like to grow up through it, to meet the people you did, to do the things you enjoyed doing. Maybe, you hope, they might even see the broader picture and the conclusions that you yourself couldn’t see at the time. This is history the way the chronicled want it to be.

Jason is nothing less than our generation’s digital historian in residence. When GeoCities went permanently offline a week ago, he was there to help preserve it for posterity.

BBS: The Documentary was a major milestone in his ongoing effort to document our digital pre-history. But it’s only the beginning; there’s also a huge documentary on text adventures, Get Lamp, that’s been in the works for a few years now. Unfortunately, progress has been slow. Because while being a digital historian is great, it’s not exactly something you get paid to do.

But maybe we can change that. Witness Jason’s kickstarter proposal:

Throughout all this, I had a day job – computer administration. It paid well, but I paid for it with my health. When my most recent employer and I parted ways, I decided I’d take this time finish some of the bigger projects I’ve been working on.

I suddenly thought back to Kickstarter and got this crazy idea – what if I simply asked the world and fans to contribute a bit of money towards keeping me somewhat solvent, and give me the opportunity to go full-time with computer history? If I was able to get all these things done over the years, what if I just asked people to subscribe or give me some patronage and in return I fill their free time with cool stuff to look at, learn from, and enjoy?

There are so many people whose online presences I greatly admire. But very few of them will go on to become part of the permanent written history of this era. I have no doubt whatsoever that Jason Scott is one of those people who will, thanks to his tireless efforts to preserve the flotsam and jetsam of our digital past, stuff that would otherwise be overlooked by the mainstream and lost forever.

I’ve pledged $100. It is an honor to support his ongoing work of preserving our shared digital pre-history. His history, is my history, is our history. A history of geeks, dorks, dweebs, nerds, and generally computer-obsessed misfits, but nonetheless — it’s something we all share.

If this is something you believe in, I urge you to pledge as well.

Go to Source