Smashing Book by Smashing Magazine

You live in Hong Kong and you are into Web Design, you may be interested in buying Smashing Book. The book is a compilation of articles from Smashing Magazine that covers graphic design, programming, but also user experience and web strategy. It looks like a very nice reference book to have by your side.

We are organizing to bulk buy a number of copies in order to enjoy some discount on the price. The more people we can gather, the better the price can be. So, if you are interested, please drop me a message below and I will add you to our list of people interested. If we can reach at least 20 people, we will order the book.

No related posts.

I finally got this opportunity back in December. Below is my a short summary of my experience with it.

Visit UserTesting.com

If I had to summarize my impressions in one word I’d choose: “Brilliant”. I was really amazed by the service offered by this company for a price that is near to nothing compared to the cost of usual research techniques.

What it is

UserTesting.com is a website that allows you to organize quick non-moderated user tests with users in the US, the UK and Australia. As a client, you basically register, enter the website you want to test, write up the tasks you want users to perform and fill in some quick user segmentation information (e.g. gender, age group, technological know-how). UserTesting will then publish your tasks to its pool of recruited users that they trained in think aloud technique. Users will in turn decide or not to take on the tasks.

The service costs $29 per user, which is a fraction of what a standard usability test costs and that includes the fee that users will receive for participating in the test.

The deliverable

What amazed me the most with this service was the speed of getting the results back. I published the test and in less than 4 hours had all 3 test videos back. Each video includes the screen recording of what the user does on the website with a synchronized audio of what they are saying (using think aloud) at that time. Video are supposed to be around 15 to 20 minutes depending on the tasks you have included in your test. In my case, videos were around 20 minutes, with one lasting 30 minutes as the user got sidetracked a bit and spend more time trying to complete the tasks.

The video deliverable is very much like a standard usability test deliverable. I found that users were pretty good at saying what they were thinking about as they were using the site. The deliverable also includes a short questionnaire, but this was quite useless in the end as the questions were too general (as of today, it is not possible to add your own questions to your study). In any case, with the speed of delivery, this opens the door to some potential extreme iterative process to improve your interface and test different design options.

What I got from it

I used the service to test a new walkthrough we implemented on GoAnimate for first time users. We included 3 users only in the study to test out the service. Basically I asked users to register on the site and head to our animation studio. Once in there our walkthrough would introduce them to the different basic feature of the studio to make sure they could start using the tool right away. I feel that this type of scenario was a great fit for using UserTesting.com. It was a straight forward linear process for users to follow that did not leave much space for users to get side-tracked. As a result, the resulting videos were really focused on the effectiveness of the interface in presenting the features of our tool.

Although all 3 users finally were able to complete all tasks, from the videos, we discovered some technical bugs that could prevent users from easily completing the walk-through. More importantly we discovered ways to improve the layout of the explanatory text and some interesting insights into how first time users might be confused by the structure and naming of things in our studio. It did not lead to any ground-breaking discovery, but did point out to really interesting issues and opened the path to more investigation into areas we had taken for granted. In this respect, the results reminded me of standard usability testing, where the results always end up revealing unexpected issues.

UserTesting vs. “Standard” Usability Testing

This said, I think I have a problem with this service taking up the name of usability testing. Personally, in almost all the usability tests I was involved in, I found that the discussion between the moderator and the user and especially the pre-test discussions always led to very interesting insights and opened new paths to improve a product. More than just testing an interface, usability tests are also an opportunity to learn more about the users and their goals. I would think that a moderated usability test that does not use this opportunity to probe more on users’ goals and motivations is missing a very important opportunity. Moderated usability testing offers great flexibility for a moderator to probe issues.

In any case, I find it very misleading to compare the price of a standard usability test and of  UserTesting’s service as they offer very different types and amounts of insights. In this sense I do not really feel UserTesting’s claim to be 1/10th of the price of competition is a valid claim. They are just not really comparable research tools. This does not change the fact though that UserTesting delivers excellent value for money.

Careful with the tasks

As I mentioned earlier, I think I used the service with a set of tasks that particularly suited the service well. It was a linear set of tasks that did not leave too many open doors for the testee to wander around. Keeping the users focused is a very important aspect of “standard” usability testing, but even more so with UserTesting.com. The user can very easily get side-tracked with an unclear or unfocused task and there will be no moderator to lead them back to the right path. This makes the service unsuitable for more holistic studies.

Conclusion

I would highly recommend UserTesting.com to anyone working on web interfaces and wanting to perform some kind of tests to gain more insights about the effectiveness of their design. It is a great and powerful tool in the hands of the right people. One needs to be very careful though that it is not a silver bullet that can replace other user research tools, and moderated usability testing especially. UserTesting.com is great to use in conjunction with other user research tools or possibly by itself for very specific design updates. Like any other research tools it can be abused, but that does not make it a lesser valuable tool anyway. A must try!

Related posts:

1. Latest Jakob Nielsen Alertbox
2. Doing Research on the Web
3. Buy on Hong Kong Now Broadband TV: Short Review

Last week (two weeks ago now) we had another meeting with fellow IxDA members. This time we tried to set up a discussion instead of just socializing randomly and it seems that it went pretty well. We split the 8 people we had in two groups and chatted for a bit. The idea of the discussion was to cover the inputs, the process and the outputs that we use in our current design-related jobs.

While the first group of 4 people focused on specifics about the processes and the deliverables used, in the group I moderated we spent most of our time exchanging stories and discussing the environment and culture in which we work here in Hong Kong.

I especially like to discuss the Hong Kong user experience environment, because we are all struggling to get people to recognize our work here. Most stories practitioners exchange are about how difficult it is to get their manager or client understand the point of interaction design or usability. So the question is: “Is there any secret weapon to be successful in such an environment?”. Well, we are all still trying to figure it out, but let me try to summarize a few points:

Cultural Differences

The first very important criteria is culture. If you are working in a very “local” (meaning Chinese I guess) company, you will have a hard time. “Local” companies have a very top-down approach to work where subordinates do not ask questions, they execute. In this environment, designing interactions, which require developing a good understanding of users and business goals, is very difficult. A tip that some of the people in the discussion raised is to play on the lack of understanding of the people around you. Lack of understanding usually creates greater freedom; so just do your job as much as possible the way you think it should be done, choose your own deliverables and focus on showing the value of your work. Do not go head on fighting against corporate culture.

Take the Time to Educate

If your company has already some understanding of the issues, then continue to educate people around you. Involve them in decision-making. Make them feel like they are making decisions themselves based on your deliverables and inputs. Work on clear deliverables that other teams can use. Education is a very slow process and can be frustrating, but it can lead to great results and get people to really see the value of interaction design or usability. Integrating your deliverables in decision making is key there… after a while other teams will request for your deliverable to make decisions.

Be ready to become the “problem solver”

Starting to ask questions is dangerous. In some cases, once other people identify you as the critical mind of the company, everything will get thrown at you. Whenever a tricky decision has to be made, you will be requested to help. You will become the person who “thinks about stuff”. Be ready to take on that role for a while at least and face the consequences. Don’t forget along the way to protect yourself and involve others in decision making… or you will not last long.

Networking

Networking in critical in all areas of business and it is even more so in Asia. A good network within or outside an organization will greatly help. Most business deals here are made with people who “trust” each others, meaning have a personal relationship. Focus on keeping good relationship will make your work easier (and harder at the same time, see paragraph above).

Anyway. The conclusion is that there is no silver bullet to solve the usability/interaction design/user experience situation in Hong Kong. From the discussion we had, corporate culture is the main factor affecting all this. As long as companies do not care much about their customers and bosses are only interested in having their ideas implemented (regardless of business sense), interaction design will not flourish fully. Let’s hope the corporate climate continues to evolve in the right direction…

Related posts:

1. Concept Models – Dan Brown's presentation at Interaction 08
2. Agile User Research
3. Usability Sense: One Control to Rule them All!?

Luckily, I found something to criticize and something that allows me discussing a bit about interface controls, an issue people take too much for granted. When designing an interface, every time there is an interaction, you need to identify the best “control” to make this interaction happen. Should I use a button? Should I use a check box? A scroll bar? Build a navigation? Or just use a plain old link? The way I see this, making this decision is both a very rigorous process and also a highly contextual one. For every kind of interaction there is a set of appropriate controls that are more effective at enabling that interaction. Depending on the rest of the interface a specific control may also be much better than another one. That is if you are not the super control that can enable any kind of interactions! Yes, there is a perfect control out there that solves all problems and looks “cool” in your interface at the same time.

The magic word ladies and gentlemen is “sliders” . Yes, sliders. I remember not so long ago designing a form for one project and one of the stakeholders told me: “Optimize it for users, but… make sure there is a slider in there as well!”. Well, turns out, sliders are not that great at solving any interaction you have to design. That’s what I’d like to highlight in this post using Newsweek website as an example. Sliders on their article page are used in two places: first, as a way to change the font size on the interface, second, in a “widget” to control the date range to retrieve popular stories. In both instances, a different kind of control may have worked better.

Font size selection is for good reasons fast becoming a must have accessibility feature on text heavy websites. The task here is simple. A user finds the font is too small. The user could be a person with a partially impaired vision for example. In this case the website offers a way to increase the size of the text font. How well does this particular control address this issue? Well, not too well. First there is clear inadequacy between the control design and the task at hand. The task addresses the need of potentially visually impaired people, but the active part of the control (the small round cursor) is so small that even a person with no visual problem will have problems clicking on the right spot. Second, does the task require the user to select a value within a wide range of options (a task the slider is good at addressing)? No. The user just wants to increase the size of the font (or reduce it maybe) and see the result right away. In that sense, why use a slider? As an interaction designer I would stick to the good old “-” and “+” button here that are much better control to address the task at hand.

What about that second slider on the interface? Well, I would argue that it does a pretty bad job as well. In a “widget” in the right column of the website, a list of popular stories is highlighted. By default this list highlight the day’s popular stories, but users (via the slider) can select articles from a different date range. Again, different problems with this control. The first is in its design. It is usually advised to display values on a slider to let users know more about the range they are choosing from. Here, there is no way before interacting with the control to know in advance which value I will be able to choose.

The second is related to the task at hand. In this respect, I do not have enough experience with publication websites to know if my point is valid or not. I am wondering what kind of range people are interested in selecting when they are looking at the most popular stories. My guess is that today’s most popular stories is good enough for most readers, but if they want to change, why limit the user to 7 days max? (the case today on the website) And why letting users select a range like the last 4 days? Maybe user research actually showed this was right, but I am not convinced it is at this point. What of three buttons or a kind of navigation that says “today”, “last week”, “last month”? Would that be enough? I am not sure, but although the slider looks cooler, in the end users will care about efficiency and effectiveness, especially on a website like Newsweek.

Related posts:

1. Usability Sense: Remember users' task
2. Usability Sense: Navigation hierarchy
3. Latest Jakob Nielsen Alertbox

The important take-away from the article for me is the need at the beginning of a project to assess what is the knowledge level of the design team and plan for research as appropriate. If the design team is representative of the target users, then little research is required at the start, but usability testing is still a good way to fine-tune the design. On the other hand, if the design project is targeting a very niche user base, then user research is necessary from the start – before any actual design happens. It takes a lot of selflessness from the team to admit that their knowledge is limited on a topic and can be tricky as well when facing a client of some sort. That is the biggest barrier in that case. The fact that good design is a highly contextual matter is still not very well understood (at least in this part of the world).

Google Search usability test
The other interesting point in Jakob Nielsen alertbox is the test they did with Google Search showing that “only” 73% of users they interviewed managed to complete a Google search. Very striking statistics indeed! Google searching for people in the Web sphere is considered the most basic of tasks and should have success rates close to 99%. So what happens there… And if some people have a hard time completing that task, consider the complexity of the tasks on your website and ponder… how many users will actually complete this…

Related posts:

1. Testing out UserTesting.com
2. Doing Research on the Web
3. Wikia Search and HAPPY NEW YEAR!

The first video I wanted to highlight is the one by Dan Brown about concept models (watch it on Brightcove or get the slides on Slideshare). That video had a big impact on me, as it made me basically put a name and reserve a space for an activity I was already performing in my projects. Without really making it formal I have been drawing concept models for most of my projects somewhere in between research (when we can actually perform any) and wireframing… or after… or at the same time…

That was the problem. Concept models had no timeframe and resources allocated to them, but somehow where necessary in order to formalize the solution that was being developed. From now on, I will dedicate specific time after research to working on these concept models whenever appropriate. I feel like a gap in my work process has been addressed here!

I have also started to read the book “Communicating Design” by Dan Brown. Although a bit too basic in some respect, it does a good job as a checklist to use during projects when you want to make sure you are on the right track and want to manage your project efficiently.

Related posts:

1. Slides on Mental Models
2. IxDA F2F: Inputs, process and outputs in interaction design
3. Clear Concept

A really good article on Personas… It is well written, entertaining and to the point; i.e. Personas are misused in most cases and a tool meant to “help companies actually get closer to their REAL customers” has been transformed into a tool used to “be perceived as close to real customers”.

Should personas be dumped altogether or should researchers educate marketing and design people about how to use them to actually make a difference in the product lifecycle (instead of just being as a marketing tool). I am for education… but how long will that process be?

Related posts:

1. The end of User-Generated Content?
2. IxDA F2F: Inputs, process and outputs in interaction design

Sourcing Information

My research process starts with a ton of RSS feeds that I have been adding over time on my personal NetVibes page. These RSS feeds being somewhat categorized, but more visually categorized in my head than on my screen. That leads to a total clutter and the impossibility for me to handle more than the current number of feeds I have.

Sharing Information

Given that I can find an article I find interesting, then I have to bookmark it and find a way to share my bookmark with others. For some reason I don’t like services like del.icio.us so I have a range of bookmarks on NetVibes, on my home FireFox, on my work FireFox and on my Yahoo Notes, etc. In the end, when I have time to post something on my blog a very small proportion of these articles I read get posted.

Tools Out There

Although I haven’t spent much time researching yet, I have been thinking about several tools I could use to improve all this content collection and communication process. One interesting tool is Zotero: this allows to bookmark not only webpages but about anything and attach notes, screenshots and more to them. It is a great data gathering tool. The problem for now is that it is not yet a sharing tool (although that should come pretty soon). Then you have readers, like Google Reader. These do a great job at organizing data and getting an RSS feed as an output. The downside is that it is less convenient to add an RSS feed to a reader rather than just bookmarking an article. Somehow the two are kind of complementary. Google Reader allows me to follow feeds and highlight the stuff I find interesting (by sharing specific articles with others for example), but it does not cater for my free browsing around the web (or even pdf files), like Zotero does.

Wordpress.com Limitations

Finally, in terms of sharing, my ideal solution would be to be able to attach to each of my blog post a feed at the end of the post with related stories (based on my personal research) that gets updated over time. Let’s say I posted two days back about Wikia Search, then at the end of that post would be an RSS feed getting information from my Google Reader for example that is related to Search Engines. That feed as I read and “flag” new articles and stories about search engines gets updated. Getting that feed could be done through Yahoo Pipes or Feed Digest. I think that would be great (although nothing really new, as this feature is on many newspaper websites already) if only I could integrate RSS Feed in my blog post on Wordpress.com.

Anyway, all this to say that so far I have not  found one tool that can answer all my needs and I am juggling with many here and there… If somebody ends up reading this post and feels like sharing tips and ideas… please do.

Related posts:

1. Latest Jakob Nielsen Alertbox
2. RSS for Everyone?
3. Riding the YouTube Wave

I am in the “very beginner” stage regarding this type of applications and to be frank I feel a bit geeky getting into it. But it seems to be that this kind of mashup applications could be changing a lot of things on the Web and letting users customize even further their web experience.

For now, I kept it simple creating two “Pipes”:

• A pipe that selects only Intermediate and Upper Intermediate lessons from my Mandarin Chinese online school instead of all lessons
• A pipe that looks for keywords like Usability, Experience and Interaction in some blogs and feeds that I read regularly

But most more can be done with these things. Ideas can be found on the Yahoo Pipes website. For more information about Yahoo Pipes and other similar application, you can read the following article from NY Times about Mashup applications.

On Digg, one can find an older post about “What is Yahoo Pipes?“. Here are some of the answers before the service was available to all:

“perhaps its something music related. maybe one of those searches based on humming a few bars”

“Can’t Think Of What It Could Possibly Be”

Despite being not too clear at first maybe, I like the name of that service. It feels like we are now able to rewire… sorry, repipe the whole Internet according to what we want.

Related posts:

1. Doing Research on the Web
2. The web auditory experience
3. Crocs, Birkenstock and … rubber boots