Ed. Note: Boing Boing’s current guestblogger Clay Shirky is the author of Here Comes Everybody: The Power of Organizing Without Organizations. He teaches at the Interactive Telecommunications Program at NYU, where he works on the overlap of social and technological networks.
Every now and again, I see a business doing something so sensible and so radical at the same time that I realize I’m seeing a little piece of the future. I had that feeling last week, after visiting my friend Scott Heiferman at Meetup.
On my way out after a meeting, Scott pulled me into a room by the elevators, where a couple of product people were watching a live webcam feed of someone using Meetup. Said user was having a hard time figuring out a new feature, and the product people, riveted, were taking notes. It was the simplest setup I’d ever seen for user feedback, and I asked Scott how often they did that sort of thing. “Every day” came the reply.
Every day. That’s not user testing as a task to be checked off on the way to launch. That’s learning from users as a way of life.
Andres Glusman and Karina van Schaardenburg designed Meetup’s set-up to be simple and cheap: no dedicated room, no two-way mirrors, just a webcam and a volunteer. This goal is to look for obvious improvements continuously, rather than running outsourced, large-N testing every eighteen months. As important, these tests turn into live task lists, not archived reports. As Glusman describes the goal, it’s “Have people who build stuff watch others use the stuff they build.”
Mark Hurst, the user experience expert, talks about Tesla — “time elapsed since labs attended” — a measure of how long it’s been since a company’s decision-makers (not help desk) last saw a real user dealing with their product or service. Measured in days, Meetup approaches a Tesla of 1.
Glusman and van Schaardenburg have also made it possible to take Jacob Nielsen’s user-testing advice — “Test with five users” — and add “…every week.” Obstacles to getting real feedback are now mainly cultural, not technological; any business that isn’t learning from their users doesn’t want to learn from their users.
On my way down after seeing the user test, the woman I’d seen on the screen got onto the elevator, and I mentioned I’d seen her trying the new interface. “Oh”, she said, surprised. “I didn’t realize anyone was actually paying attention to me.”
Hurst: Time elapsed since labs attended | Nielsen: Why You Only Need to Test With 5 Users