I finally got this opportunity back in December. Below is my a short summary of my experience with it.

Visit UserTesting.com

If I had to summarize my impressions in one word I’d choose: “Brilliant”. I was really amazed by the service offered by this company for a price that is near to nothing compared to the cost of usual research techniques.

What it is

UserTesting.com is a website that allows you to organize quick non-moderated user tests with users in the US, the UK and Australia. As a client, you basically register, enter the website you want to test, write up the tasks you want users to perform and fill in some quick user segmentation information (e.g. gender, age group, technological know-how). UserTesting will then publish your tasks to its pool of recruited users that they trained in think aloud technique. Users will in turn decide or not to take on the tasks.

The service costs $29 per user, which is a fraction of what a standard usability test costs and that includes the fee that users will receive for participating in the test.

The deliverable

What amazed me the most with this service was the speed of getting the results back. I published the test and in less than 4 hours had all 3 test videos back. Each video includes the screen recording of what the user does on the website with a synchronized audio of what they are saying (using think aloud) at that time. Video are supposed to be around 15 to 20 minutes depending on the tasks you have included in your test. In my case, videos were around 20 minutes, with one lasting 30 minutes as the user got sidetracked a bit and spend more time trying to complete the tasks.

The video deliverable is very much like a standard usability test deliverable. I found that users were pretty good at saying what they were thinking about as they were using the site. The deliverable also includes a short questionnaire, but this was quite useless in the end as the questions were too general (as of today, it is not possible to add your own questions to your study). In any case, with the speed of delivery, this opens the door to some potential extreme iterative process to improve your interface and test different design options.

What I got from it

I used the service to test a new walkthrough we implemented on GoAnimate for first time users. We included 3 users only in the study to test out the service. Basically I asked users to register on the site and head to our animation studio. Once in there our walkthrough would introduce them to the different basic feature of the studio to make sure they could start using the tool right away. I feel that this type of scenario was a great fit for using UserTesting.com. It was a straight forward linear process for users to follow that did not leave much space for users to get side-tracked. As a result, the resulting videos were really focused on the effectiveness of the interface in presenting the features of our tool.

Although all 3 users finally were able to complete all tasks, from the videos, we discovered some technical bugs that could prevent users from easily completing the walk-through. More importantly we discovered ways to improve the layout of the explanatory text and some interesting insights into how first time users might be confused by the structure and naming of things in our studio. It did not lead to any ground-breaking discovery, but did point out to really interesting issues and opened the path to more investigation into areas we had taken for granted. In this respect, the results reminded me of standard usability testing, where the results always end up revealing unexpected issues.

UserTesting vs. “Standard” Usability Testing

This said, I think I have a problem with this service taking up the name of usability testing. Personally, in almost all the usability tests I was involved in, I found that the discussion between the moderator and the user and especially the pre-test discussions always led to very interesting insights and opened new paths to improve a product. More than just testing an interface, usability tests are also an opportunity to learn more about the users and their goals. I would think that a moderated usability test that does not use this opportunity to probe more on users’ goals and motivations is missing a very important opportunity. Moderated usability testing offers great flexibility for a moderator to probe issues.

In any case, I find it very misleading to compare the price of a standard usability test and of  UserTesting’s service as they offer very different types and amounts of insights. In this sense I do not really feel UserTesting’s claim to be 1/10th of the price of competition is a valid claim. They are just not really comparable research tools. This does not change the fact though that UserTesting delivers excellent value for money.

Careful with the tasks

As I mentioned earlier, I think I used the service with a set of tasks that particularly suited the service well. It was a linear set of tasks that did not leave too many open doors for the testee to wander around. Keeping the users focused is a very important aspect of “standard” usability testing, but even more so with UserTesting.com. The user can very easily get side-tracked with an unclear or unfocused task and there will be no moderator to lead them back to the right path. This makes the service unsuitable for more holistic studies.

Conclusion

I would highly recommend UserTesting.com to anyone working on web interfaces and wanting to perform some kind of tests to gain more insights about the effectiveness of their design. It is a great and powerful tool in the hands of the right people. One needs to be very careful though that it is not a silver bullet that can replace other user research tools, and moderated usability testing especially. UserTesting.com is great to use in conjunction with other user research tools or possibly by itself for very specific design updates. Like any other research tools it can be abused, but that does not make it a lesser valuable tool anyway. A must try!

Related posts:

1. Latest Jakob Nielsen Alertbox
2. Doing Research on the Web
3. Buy on Hong Kong Now Broadband TV: Short Review

There are two actors in this interaction: The reader and the author.

The reader, browsing SlideShare (or more probably searching on Google) for content related to his study or presentation, finally finds something on SlideShare inside a presentation. Too bad though, the  author of the presentation decided he would not allow readers to download the presentation.

SlideShare has well addressed this need in their design. In case a presentation can be downloaded, the interface displays a clear “Download” button just on top of their slideshow, and if the presentation is not downloadable then the button is changed into a “Request Download” one.

SlideShare

SlideShare Messaging Form

Not only would it be more efficient, it would also add better control when the other actor in this interaction takes over. The  author is informed that somebody sent him a message, but no mention is made regarding the fact that the reader wants to download his presentation, that is unless the reader took the time to write a clear message explaining exactly what is needed.

… And even if they do, the  author is then faced with the task of turning the download option on for the presentation. As the original flow is related to “request download” one could expect that the option to turn this on would be clearly highlighted on the interface next to the message, but nothing like this is offered to the  author who if he is not using SlideShare on a regular basis (like me) is left wandering around the options of the presentation and missing the little checkbox on that page several times before being able to complete the task.

SlideShare Private Message

To go a bit further, we may also consider that if the  author made his presentation “non-downloadable” he or she may have had reasons for doing so. In that respect it may be useful to offer the possibility to the  author to let another specific user to download the presentation rather than having to allow anyone to download the presentation.

Oh. No! It sounds like my entirely objective blog post became a rant… S … Sorry for that. The aim is just to point out that a flow has to be thought from beginning until the end in order to be successful (including all actors in that interaction). A button on an interface is not enough in most cases… And thus now I will look back at the feature we have been building on GoAnimate and see where we may have fell into this trap as well (which with daily turnarounds like in any start-ups is unavoidable)

Merry Christmas and Happy New Year

Related posts:

1. Usability Sense: One Control to Rule them All!?
2. Slides on Mental Models
3. Buy on Hong Kong Now Broadband TV: Short Review

Last week (two weeks ago now) we had another meeting with fellow IxDA members. This time we tried to set up a discussion instead of just socializing randomly and it seems that it went pretty well. We split the 8 people we had in two groups and chatted for a bit. The idea of the discussion was to cover the inputs, the process and the outputs that we use in our current design-related jobs.

While the first group of 4 people focused on specifics about the processes and the deliverables used, in the group I moderated we spent most of our time exchanging stories and discussing the environment and culture in which we work here in Hong Kong.

I especially like to discuss the Hong Kong user experience environment, because we are all struggling to get people to recognize our work here. Most stories practitioners exchange are about how difficult it is to get their manager or client understand the point of interaction design or usability. So the question is: “Is there any secret weapon to be successful in such an environment?”. Well, we are all still trying to figure it out, but let me try to summarize a few points:

Cultural Differences

The first very important criteria is culture. If you are working in a very “local” (meaning Chinese I guess) company, you will have a hard time. “Local” companies have a very top-down approach to work where subordinates do not ask questions, they execute. In this environment, designing interactions, which require developing a good understanding of users and business goals, is very difficult. A tip that some of the people in the discussion raised is to play on the lack of understanding of the people around you. Lack of understanding usually creates greater freedom; so just do your job as much as possible the way you think it should be done, choose your own deliverables and focus on showing the value of your work. Do not go head on fighting against corporate culture.

Take the Time to Educate

If your company has already some understanding of the issues, then continue to educate people around you. Involve them in decision-making. Make them feel like they are making decisions themselves based on your deliverables and inputs. Work on clear deliverables that other teams can use. Education is a very slow process and can be frustrating, but it can lead to great results and get people to really see the value of interaction design or usability. Integrating your deliverables in decision making is key there… after a while other teams will request for your deliverable to make decisions.

Be ready to become the “problem solver”

Starting to ask questions is dangerous. In some cases, once other people identify you as the critical mind of the company, everything will get thrown at you. Whenever a tricky decision has to be made, you will be requested to help. You will become the person who “thinks about stuff”. Be ready to take on that role for a while at least and face the consequences. Don’t forget along the way to protect yourself and involve others in decision making… or you will not last long.

Networking

Networking in critical in all areas of business and it is even more so in Asia. A good network within or outside an organization will greatly help. Most business deals here are made with people who “trust” each others, meaning have a personal relationship. Focus on keeping good relationship will make your work easier (and harder at the same time, see paragraph above).

Anyway. The conclusion is that there is no silver bullet to solve the usability/interaction design/user experience situation in Hong Kong. From the discussion we had, corporate culture is the main factor affecting all this. As long as companies do not care much about their customers and bosses are only interested in having their ideas implemented (regardless of business sense), interaction design will not flourish fully. Let’s hope the corporate climate continues to evolve in the right direction…

Related posts:

1. Concept Models – Dan Brown's presentation at Interaction 08
2. Agile User Research
3. Usability Sense: One Control to Rule them All!?

Luckily, I found something to criticize and something that allows me discussing a bit about interface controls, an issue people take too much for granted. When designing an interface, every time there is an interaction, you need to identify the best “control” to make this interaction happen. Should I use a button? Should I use a check box? A scroll bar? Build a navigation? Or just use a plain old link? The way I see this, making this decision is both a very rigorous process and also a highly contextual one. For every kind of interaction there is a set of appropriate controls that are more effective at enabling that interaction. Depending on the rest of the interface a specific control may also be much better than another one. That is if you are not the super control that can enable any kind of interactions! Yes, there is a perfect control out there that solves all problems and looks “cool” in your interface at the same time.

The magic word ladies and gentlemen is “sliders” . Yes, sliders. I remember not so long ago designing a form for one project and one of the stakeholders told me: “Optimize it for users, but… make sure there is a slider in there as well!”. Well, turns out, sliders are not that great at solving any interaction you have to design. That’s what I’d like to highlight in this post using Newsweek website as an example. Sliders on their article page are used in two places: first, as a way to change the font size on the interface, second, in a “widget” to control the date range to retrieve popular stories. In both instances, a different kind of control may have worked better.

Font size selection is for good reasons fast becoming a must have accessibility feature on text heavy websites. The task here is simple. A user finds the font is too small. The user could be a person with a partially impaired vision for example. In this case the website offers a way to increase the size of the text font. How well does this particular control address this issue? Well, not too well. First there is clear inadequacy between the control design and the task at hand. The task addresses the need of potentially visually impaired people, but the active part of the control (the small round cursor) is so small that even a person with no visual problem will have problems clicking on the right spot. Second, does the task require the user to select a value within a wide range of options (a task the slider is good at addressing)? No. The user just wants to increase the size of the font (or reduce it maybe) and see the result right away. In that sense, why use a slider? As an interaction designer I would stick to the good old “-” and “+” button here that are much better control to address the task at hand.

What about that second slider on the interface? Well, I would argue that it does a pretty bad job as well. In a “widget” in the right column of the website, a list of popular stories is highlighted. By default this list highlight the day’s popular stories, but users (via the slider) can select articles from a different date range. Again, different problems with this control. The first is in its design. It is usually advised to display values on a slider to let users know more about the range they are choosing from. Here, there is no way before interacting with the control to know in advance which value I will be able to choose.

The second is related to the task at hand. In this respect, I do not have enough experience with publication websites to know if my point is valid or not. I am wondering what kind of range people are interested in selecting when they are looking at the most popular stories. My guess is that today’s most popular stories is good enough for most readers, but if they want to change, why limit the user to 7 days max? (the case today on the website) And why letting users select a range like the last 4 days? Maybe user research actually showed this was right, but I am not convinced it is at this point. What of three buttons or a kind of navigation that says “today”, “last week”, “last month”? Would that be enough? I am not sure, but although the slider looks cooler, in the end users will care about efficiency and effectiveness, especially on a website like Newsweek.

Related posts:

1. Usability Sense: Remember users' task
2. Usability Sense: Navigation hierarchy
3. Latest Jakob Nielsen Alertbox

The important take-away from the article for me is the need at the beginning of a project to assess what is the knowledge level of the design team and plan for research as appropriate. If the design team is representative of the target users, then little research is required at the start, but usability testing is still a good way to fine-tune the design. On the other hand, if the design project is targeting a very niche user base, then user research is necessary from the start – before any actual design happens. It takes a lot of selflessness from the team to admit that their knowledge is limited on a topic and can be tricky as well when facing a client of some sort. That is the biggest barrier in that case. The fact that good design is a highly contextual matter is still not very well understood (at least in this part of the world).

Google Search usability test
The other interesting point in Jakob Nielsen alertbox is the test they did with Google Search showing that “only” 73% of users they interviewed managed to complete a Google search. Very striking statistics indeed! Google searching for people in the Web sphere is considered the most basic of tasks and should have success rates close to 99%. So what happens there… And if some people have a hard time completing that task, consider the complexity of the tasks on your website and ponder… how many users will actually complete this…

Related posts:

1. Testing out UserTesting.com
2. Doing Research on the Web
3. Wikia Search and HAPPY NEW YEAR!

The navigation is arguably the most important area in a website. From the navigation, the user should be able at a glance to create a mental model (definition on Wikipedia) of what the website is about and what it has to offer (be it content, services or products).

That’s where Information Architecture comes in. Information Architecture is basically two things: labeling and grouping information.

Labeling
Labeling seems straight-forward at first, but is a quite political issue. Good labeling should be clear and intuitive, meaning that when the user reads the name of a section, he or she should be able to guess what kind of content is included in it. The usual conflict when it comes to labeling is with the marketing department. In terms of branding and “sounding cool” many marketers come up with fancy names for sections they want on the website. Problem is,… these names require more effort on the user’s side to learn and thus affect usability. Not that they are always bad. If a “label” is consistently used throughout an organization regardless of the channel (online or offline), then it could work. As usual, the answer to that problem is: it depends on the context of use (like most usability problems).

An example with unclear naming… good or bad?!

Grouping
Grouping is the second aspect of information architecture. Grouping relates to how the content is put together to form sections, sub-sections, etc. Grouping is really key in supporting users in building their mental model of the website. A good grouping is hierarchical by nature and this hierarchy should be reflected in the navigation of the website. It is usually good to have a primary grouping where each piece of content resides in a single group and then if needed build multiple ways to navigate that information from different angles. Below screenshot shows this. On the International Herald Tribune, content is organized by topics first, but then it is also possible to browse according to location as well.

But that’s where the problems start (and where the actual blog post I meant to write start as well). The way the grouping on IHT is displayed really does not do a good job at helping users build a clear mental model. The design of the navigation creates the impression that regions (americas, europe,…) are subsections of the main iht.com section (that appears selected). That could work, but let’s look at what happens when we decide to look at business articles…

As the user is interacting with the navigation area, this same area should provide a feedback of the action that took place. Unfortunately at this stage, nothing happened. The homepage (iht.com) still looks selected, what looked like sub-sections (americas, europe,…) did not change. The mental model that users had build based the homepage does not hold anymore. Of course, the content below changed, and the page has a title “Business with Reuters”, but the job is not done. Users are also not given the opportunity to drill down more into sub-sections under business (possible at the bottom of the page), which I would say is a pretty important features.

Anyway. My post is pretty long already. To sum up: A navigation should support users in building a mental model of the website. That mental model should be reinforced as they browse to different pages. Easier said than done…

Related posts:

1. Usability Sense: One Control to Rule them All!?
2. Usability Sense: Remember users' task
3. The end of User-Generated Content?

For a long time, the registration process on websites has been a hindrance to letting users complete their task. Until Amazon came out, users could not do anything before they registered, resulting in many lost opportunities for websites. Indeed, without a teaser into what users can get, these usually do not want to create a profile. The rule is “show me what I get and maybe I will create a profile”. Creating a profile is a trade off, where the user is trading personal information against a (usually free) service.

So, Amazon came and now users could start looking for books online and only create a profile at the check out time. Everything got solved right?! Well, not really. Many websites still fail to integrate the registration (or login) process well. This usually means that users who did not register reach a stage where they have to register. Upon creating their profile, they have to restart their process from scratch or more often extra steps are added to their initial task.

There the simple usability concept is “remember the user’s task”. An example of a good experience I just had this morning (thus this post) is with DocStoc. I went there looking for a document. I found it and upon clicking the download button, I got sent to a registration page. At this stage, I pondered. Is what DocStoc offers enough for me to provide me personal information. I thought so at that time and went on to create my profile.

After two pages filling data, I saved my profile and the document I requested got downloaded automatically. Creating my profile, added a few steps to my task, but in this end did not affect my original task.

Common Sense many would say… yes, but oftentimes common sense is exactly what websites lack.

Related posts:

1. Usability Sense: One Control to Rule them All!?
2. Usability Sense: Navigation hierarchy
3. Manpower website

Sourcing Information

My research process starts with a ton of RSS feeds that I have been adding over time on my personal NetVibes page. These RSS feeds being somewhat categorized, but more visually categorized in my head than on my screen. That leads to a total clutter and the impossibility for me to handle more than the current number of feeds I have.

Sharing Information

Given that I can find an article I find interesting, then I have to bookmark it and find a way to share my bookmark with others. For some reason I don’t like services like del.icio.us so I have a range of bookmarks on NetVibes, on my home FireFox, on my work FireFox and on my Yahoo Notes, etc. In the end, when I have time to post something on my blog a very small proportion of these articles I read get posted.

Tools Out There

Although I haven’t spent much time researching yet, I have been thinking about several tools I could use to improve all this content collection and communication process. One interesting tool is Zotero: this allows to bookmark not only webpages but about anything and attach notes, screenshots and more to them. It is a great data gathering tool. The problem for now is that it is not yet a sharing tool (although that should come pretty soon). Then you have readers, like Google Reader. These do a great job at organizing data and getting an RSS feed as an output. The downside is that it is less convenient to add an RSS feed to a reader rather than just bookmarking an article. Somehow the two are kind of complementary. Google Reader allows me to follow feeds and highlight the stuff I find interesting (by sharing specific articles with others for example), but it does not cater for my free browsing around the web (or even pdf files), like Zotero does.

Wordpress.com Limitations

Finally, in terms of sharing, my ideal solution would be to be able to attach to each of my blog post a feed at the end of the post with related stories (based on my personal research) that gets updated over time. Let’s say I posted two days back about Wikia Search, then at the end of that post would be an RSS feed getting information from my Google Reader for example that is related to Search Engines. That feed as I read and “flag” new articles and stories about search engines gets updated. Getting that feed could be done through Yahoo Pipes or Feed Digest. I think that would be great (although nothing really new, as this feature is on many newspaper websites already) if only I could integrate RSS Feed in my blog post on Wordpress.com.

Anyway, all this to say that so far I have not  found one tool that can answer all my needs and I am juggling with many here and there… If somebody ends up reading this post and feels like sharing tips and ideas… please do.

Related posts:

1. Latest Jakob Nielsen Alertbox
2. RSS for Everyone?
3. Riding the YouTube Wave

A new shopping concept has hit town!

A PCCW service: now shop (channel 501) offers you local and Hollywood hottest DVDs, top-selling books, console games, electronics , lifestyle products and other services including movie ticketing and the No.1 Shopping Gallery. The new Stock Market Express, the new pay service providing Real-time stock quotes , FX spot rates and Gold prices for you to grasp every investment opportunity.

I have been taking a few pictures of the interface (accessible on Flickr)and I prepared a small presentation where I put my thoughts about the service. I also reviewed some elements of the interface from a usability perspective.

I prepared all this pretty quickly and did not do research about similar services in other countries. It is quite rough, but I hope it can be interesting to some and a basis for discussion on the topic of buying through interactive TV.

[slideshare id=124500&doc=hong-kong-now-tv-shop4606&w=425]

Let me know your thoughts…

Related posts:

1. Service level – Comparison between Hong Kong and France
2. 41st Hong Kong Brands and Products Expo
3. Web Wednesday Hong Kong

I put quotes around  “upset” as it may be too strong a word. In customer experience everything is about emotions, and feelings that experiences trigger. In this case, a simple confirmation page did the trick. I felt treated like a batch of files to process and almost teased at. (See picture)

I registered my profile and uploaded my resume on Manpower. After submitting my information, here comes the confirmation page: “Congratulations, Preferred Candidate!”

Everything in this line is wrong:

1. I just submitted my personal information and I am still referred as a “Preferred Candidate”. Why don’t they use my name?
2. The word “Preferred” is somewhat strange, as every single candidate registering on the Web site is “preferred”. What value does that provide me?
3. The word “Congratulations” is really too much! My task, my aim is to look for a job. I get congratulated because I created an account? What’s to be happy about that? Just get over that account creation and show me some job openings. Or should I be contempt with the fact that I created an account?

Just in one sentence, the company manages to show very clearly their disconnect with candidates and the fact that candidates are treated just like files and not people, which maybe is what Manpower is about in a way.

This is just an example of a problem on their websites. I could go on with the “Next page” button on top of the page when doing a search ( it took me three searches to see it), and the lack of posting date for ads in the search (an information crucial to the candidate).

Customer experience is sometimes (often?) created out of the small things. Especially when your customer is carrying a heavy luggage into the experience (for example, candidates may be quite low as their job search drags on and are quite sensitive), tone and phrasing is very important in order to build a positive experience.

Related posts:

1. Wikia Search and HAPPY NEW YEAR!
2. Forced opt-in by the French government