Choosing a Platform

Tools such as Figma or Invision can easily simulate the functions of a “clickable” user interface but are incapable of interactions by voice. We needed a new tool that would allow users to actually speak to and hear audio feedback from our application. Based on the available prototyping tools with voice capability, we considered using Adobe XD and Voiceflow.  

Adobe XD has powerful UI and visual design capabilities with basic voice prototyping functions.

Adobe XD is a full-fledged vector-based design application that supports voice prototyping, while Voiceflow has very limited visual editing capabilities, but is a very powerful voice prototyping tool. We assumed that our needs would be almost entirely voice-oriented, so the decision was made to proceed with Voiceflow.

The Voiceflow builder has an easy drag and drop capability to quickly construct voice interactions.

Voiceflow also had several features that helped cement it as our top choice. Prototypes are easily shareable and can be tested by anyone with almost no setup, as prototypes are available right in your browser, and can be shared with a hyperlink. Another feature that stood out to us was the expertise built right into the tool, meaning, there are best practice tips and tricks available right at your fingertips. This was not as robust with Adobe XD, as that tool isn’t solely focused on voice design. Voiceflow also had incredible levels of customization for your voice assistant – you could select different voice tones, pitches, and most importantly speeds. It also supports variables – for example, you could tell the prototype your name, and it would remember it and say it back to you.

Defining The Problem

Any good UX’er that’s worth their salt knows that you can’t create a solution without fully understanding the problem. While there are many ways to do this, we started our process with two workshops with some of our users and subject matter experts to learn what workflows pose the biggest challenges to them. We then assessed if those flows could be simplified with voice commands. Those two separate paths were then compared using a workflow design tool called Whimsical. The most important thing to remember during these workshops was two ensure that the tasks we were simplifying were truly meaningful tasks, and not simple solutions such as “open email,” which would actually be easier to do with a couple of taps.

An example of a Whimsical workflow

Prototyping Time!

Voiceflow has a pretty neat onboarding experience where they have a full prototype set up to show you the basics. Voiceflow is pretty feature-rich, but our prototype focused primarily on using “choice” blocks, visuals, and audio.

Recording of our voice prototype created in Voiceflow.

Choice Blocks

This is the backbone of our prototypes. Basically, you can type out what you’re going to say, and set up what block you would like to go to when that happens. For those code-inclined, think of it like an if-then statement. In fact, the core of our entire prototype is based around one block with nine “choices” stemming from it. This pattern is particularly useful if your application or site has one main “hub” where most actions derive from.

An example of a “Hub”

In our prototype, we had several of the workflows we previously mentioned start from this hub and almost always returned here once the workflow was completed. The “choices” that you can speak in these instances are what Voiceflow calls “utterances”. An example of a common utterance would be “What’s the weather?”.

Any of these utterances above signify you’d like the second choice in a list that is being displayed on the screen.

Another extremely important part of choice blocks is “throwaway text”. Throwaway text are wildcards that exist in your utterances. If we were to apply this logic to the weather example above, it would look something like this in Voiceflow: “{throwaway text} weather”. In this case, any time you end what you’re saying with the word “weather”, it would send you down that utterance path. This can be extremely useful when you have words that Voiceflow struggles to recognize. In our case, the application we were working with uses complex medical terminology and sentences that would trip up any out-of-the-box voice solution. It also made demos go much smoother.

Occasionally the Voiceflow would get tripped up with “adding protocols,” so we implemented throwaway text.

Be careful with throwaway text, however. You should always ensure that whatever you prototype can work effectively in your application with adequate voice model training.

Visuals

While Adobe XD is far superior for visual prototyping, you can get away with more than you might expect in Voiceflow… You just need to think of each visual you’d like to see, and set a time delay in between blocks. For example, we have a workflow to open our application’s camera. To complete the first step in this workflow, you hold the spacebar to allow Voiceflow to “listen” and say our application’s name to “wake” our virtual assistant. There is then a visual where a speech bubble appears on the screen that says “I’m listening”.

After the application is “listening,” you hold the spacebar again and say “open camera”. Voiceflow will then listen to your command and if it recognizes it, you will cycle through the remaining images to get to the next “choice” block.

Audio

Audio had a couple of primary uses in our prototype. The first being a signifier that the application did in fact hear your wake command. It was a simple free sound clip that I found online that wasn’t jarring or off-putting, since you’d be hearing it every time you wanted to wake our assistant. The other use cases primarily consisted of confirmations, such as a setting being saved or a message being sent. In use-cases like this, having an assistant say a full command would be a bit excessive and wildly irritating after several times. Think about if every time you received a text message, your phone said “you have a message”.

Iterating

Throughout the whole prototyping process, countless iterations were done. Every time a workflow was completed, it was tested, and periodically workflows were shown to different doctor-employees and various other stakeholders. There were different versions created for different specialties, voice responses were sped up, and unnecessary screens were dropped. After all, one of the main appealing features of voice is to speed up peoples’ workflows and reduce the number of clicks needed to accomplish tasks.

The Experience

As a UX professional, creating a voice prototype with Voiceflow was an interesting challenge. You need to handle similar problems to regular design (what happens if someone clicks this thing they aren’t supposed to VS what happens when someone says this thing they aren’t supposed to), but there are also some differences. There are no pixels or colors, so things don’t have to be perfect in that sense, but there are also limited standards or conventions, so you really have to focus on what your user’s needs and pain points are, rather than what industry heuristics may be.

Demoing Tips

Considering most companies are using voice prototyping software like Voiceflow as a proof of concept, you’re probably going to be demoing it… a lot. Throughout the multitude of demos I’ve conducted for user feedback, here are a few tips:

For visuals, use small images.

If you are using visuals, and you probably should be, use as small as possible images while maintaining visual fidelity to ensure your ability to demo isn’t hampered by large prototypes. I typically designed images at full size but exported them as .5x their original size.

Have only your Voiceflow demo and script open.

Shut down everything else you have open while demoing. Yes, that means chrome with your 10,153 tabs open. It will greatly reduce your demo lag. I would still recommend leaving your script tab or program open as well. This leads me to my next point…

Have a well-thought-out, word-for-word script.

As a UX researcher, I know that flexibility is paramount during a session, but in my opinion, having a word-for-word script of the key points of your interaction is incredibly important. An entire interview can be destroyed if you get flustered enough and lose your place. The same goes for a voice demo… You only get one shot at a first impression.

Speak clearly and don’t rush.

The voice recognition in Voiceflow is pretty good, but it still gets tripped up on uncommon words (if I could wipe the word “intramail,” from existence, I would). So make sure you are speaking naturally, but as clearly as possible.

Use wildcards or “throwaway text”.

No matter how good the voice recognition software is, chances are it isn’t waiting on an entire phrase to be matched. It most likely is using keywords and your prototype shouldn’t be any different. Many phrases can be complete, but for others, it makes sense just to have keywords and throwaway text as long as there are enough keywords to not confuse the voice prototyping program.

Conclusion

Since voice technology is still in its infancy, the only standards that exist are simple commands like “read my messages,” or “navigate to…”, but at my company, we’re trying to ensure that we’re solving real challenges, and not simply adding voice as an “us too” feature. This makes it even more important that we align with our user’s mental models but in a different way. In one of our workshops, we even said to the participants that they should pretend we are a human assistant. Would you ask your human assistant to send a message for you? No, you’d do it yourself because it’s just as easy. However, a task like “order prescription X for this patient, 100mg,” can be incredibly impactful because of the huge amount of clicks and typing that you are saving.

When you shop for a new basketball, you might use multiple surfaces i.e. the phone to quickly browse the best basketball, the website to view details, and the store to eventually purchase. Similarly, enterprise users building or managing applications or services like a shopping website – also use multiple surfaces to complete their goal of building the site.

We learned early on that switching between multiple surfaces is a natural behavior for enterprise users however, the key question we wanted to answer was when and why do users choose between these surfaces. So we performed research to learn about user behavior and also created a framework to help teams across the company prioritize which surfaces to build for. The following outlines our research process and lessons learned along the way.

What do we mean when we say “surfaces”?

Graphical User Interface (GUI)

Users interact with the system or application via visualizations, icons, and graphics rather than code. Your desktop is a form of a GUI. For example, if you need to create a folder on your desktop, it includes a series of button clicks on the desktop to search the options, ‘Add folder’ and lastly enter a new name for the folder. GUI is ideal for learning a new environment as it’s easy to walk users through complex workflows and it’s great to provide visual cues for task completion. Furthermore, GUI enables visualization of graphs, metrics and reports, making it a great UI for monitoring tasks.

Command Line Interfaces (CLI)

Users use a terminal or a script that contains lines of short commands to use a system or an application. CLIs are incredibly useful when you need to run the same command multiple times. Taking the previous example if you needed to create 100 folders on your desktop, it might take you a long time to repeat the actions on a GUI 100 times whereas a CLI makes it easier as you can quickly write a script where a command for creating a folder is called 100 times by the system thereby automating the process for you.  As a result, you will find that development environments typically consist of CLIs.

Application Programming Interfaces (API)

It’s software that allows two applications to talk to each other through code. If you have used a car ride service like Lyft or Uber, you must have come across the maps features that are rendered through a bunch of API calls made to map rendering services like Google Maps or OpenStreetMaps.

Infrastructure as Code (IaC)

Lastly, IaC consists of tooling that allows users to write configurations (a.k.a config files) that automate infrastructure operations. For example, a growing startup that is running its services on Google Cloud, and after a year due to the growing user base they want to expand to Amazon Web Services (AWS) in which case, they have to rewrite all their infrastructure operations to work on AWS. This requires a huge manual effort and this is where IaC makes their life simpler. It helps users to use templates to automate a lot of the functions from API and CLI’s such that the code can easily be made to work across different infrastructure platforms thereby making it easy to handle operations across multiple platforms. Terraform and Ansible are examples of such configs written in a domain-specific language.

The research question

While building user experiences for users, we want to provide the users a  “happy path” i.e. reduce the number of steps taken on an interface to complete a goal. As a result, we might consider context switches between interfaces as a suboptimal experience, however, our research on enterprise users using cloud products found that it is natural for our users to switch between multiple surfaces to complete a single goal (as shown in the diagram below).

The important question we wanted to answer through our research is when and why do users choose to use different surfaces and understand how we can provide a seamless experience between the surfaces for our users.

Fig 1: An example of a UX Research participant navigating through different surfaces to complete a goal in one of the Cloud products. Each number is a click towards the surface.

UX Research Process

To understand how users use these surfaces to complete goals, we analyzed dozens of research to develop insights and a framework.

Step 1: Literature review

This work is built on 30 internal studies conducted by UX researchers, across 8 different cloud products (GKE, GCP, Anthos, Rancher, Terraform, etc) involving methodologies like user interview, usability test, cognitive walkthrough, survey, CUJ analysis, and quantitative data analysis. In addition to consolidating their work, we also analyzed external articles & academic papers on HCI theory to generate initial insights.

Step 2: Task mapping

The next step was to better understand how users traverse surfaces in the real world. We each examined a few common user journeys which helped us distinguish between how users use surfaces to complete different goals

Step 3: Develop the framework

Using the insights from the above steps, we created a decision tree to highlight the factors that influence users to decide on surfaces to use. By using this method, we were able to operationalize and test our insights. This map was the foundation of our framework.

The output: A scalable framework

We started this work to demystify how our users use different surfaces to reach a goal. We launched this research to understand how users use surfaces today and to help teams figure out which ones to use – and at what point – when building their products. As a result, we created a framework that provides guidance to help any team make a decision on which surfaces to focus on for their products.

The framework has three parts:

1. User goals

There are 11 goals that we identified for when our users use our product. They included generic goals like learning, discovering, onboarding, automating, etc. We matched goals to common surfaces used when completing the goal. 

2. User proficiency

Determines the level of the domain (technical knowledge) and product expertise of users. We categorized user expertise into 3 segments.

3. Product maturity

Depending on the product’s maturity (Beta, Private Preview, General Availability), different surfaces might be prioritized. If it’s a new product, there is more of a need to build surfaces such as GUI that help people learn, versus more mature products where users might want more automation capabilities such as API or IaaC.

Insights

From this research, we learned a bit about enterprise user behavior:

Surfaces complement each other: It is natural for Enterprises customers to rely on suite of surfaces to accomplish their goals

Similar to how you might use a phone, laptop, and go into the store to buy an item, enterprise customers similarly use multiple touchpoints to complete a goal. The difference is they rely on multiple surfaces in small time frames to complete a goal. This could include a CLI to write a command and then validate it in a GUI. If users have trouble accessing one from the other – or if they are too different – or if the terminology is different, this could mean a frustrating experience for the customer

Users choice of the suite of surfaces is dependent on their goal (i.e. type of task they intend to do), user proficiency, and product maturity

These 3 variables help determine which surface to prioritize building. For example, if a new product is for a new user who is an expert, then it would be worth prioritizing GUI, but providing an avenue for the user to learn more advanced, automated surfaces to grow into. On the other hand, if you have an existing product for existing and expert users, then you’ll need to prioritize automation technologies like CLI and API.

Expert users when using a new tool/software rely on extra guidance or a GUI interface

When we started this work, there was an assumption that expert users would only use automation technologies. Through this research, we debunked that myth. It turns out expert users use GUI heavily to learn about a new product and validate even if they are using automated tools to complete actions. We also found that while expert users decrease their usage of GUI as time goes on, many still use it for day-to-day tasks. Therefore, making sure you have a GUI interface to pair with your CLI or API is important – and it may be worth discovering what users like about GUI and factoring that into non-GUI surfaces.

Best practices and lessons learned from this project

If you work on a product that offers multiple surfaces for its users, here are some things we learned from our experience, which could be helpful for you or your team:

Conclusion

By conducting this research, we were able to understand how enterprise users use different surfaces to complete their goals and influence internal teams to apply this knowledge to their product areas. As outlined above, it is natural for our users to switch between surfaces. Ultimately the user is trying to decrease the effort & risk associated with a task and will choose a ‘suite of surfaces’ that help them complete their goal faster. As a result, researchers, designers, or product owners ought to understand user behaviors across surfaces and enable seamless transitions between the surfaces to improve the overall user experience. We hope the insights and approaches we shared, helped you understand the factors motivating users to choose multiple surfaces and piqued your interest in understanding cross-interface usage for your products.

Acknowledgments

Sarah D’Angelo‎, Spencer Sugarman, Marc Fong, Andy Quin, Jason Dong, Brian Grant, Anna Wu, Jason Schwarz, Mackenzie Sunday, Alec Scharff, Hadar Scharff, Ekaterina Koroleva, Andy Qin‎, Julie Zhuying Li‎, , Santosh Mathan‎, Yann Riche, Jess Tsimeris‎ , Arunima Kashyap, Andrew Macvean, Kelli Kludt, Monica Caraway, Maks Osowski, Marian Kędzierski, Marina Masaki, Julianne Bryant

At UXPressia, we provide an online customer journey mapping tool for global audiences, so it’s only logical that we usually conduct user interviews online. When you know what to ask, how, and when, an online interview can be as informative as an in-person meeting.

Without further ado, let’s go step by step throughout the entire process of conducting a remote user interview to help you make it more efficient and maximize the impact of your research.

Before the interview

Preparation is key. The better you prepare (select respondents, develop questions, choose tools) for the interview, the better results you’ll get. 

Identifying the right respondents

Recruiting the right participants is the first and perhaps the most important step, this is why many find the services of companies like the Perelson’s Utah County staffing agency very usefull. No matter how well your interview is structured, it won’t bring you the results you need if you focus on the wrong audience.

Depending on the objectives, we usually recruit users who can provide valuable insights on the topic interesting to us. Important note: sometimes you have to look around and ask yourself who else can give you the information you need.

For example, if you want to research the problem of user churn, it’s not enough to interview just those who left. Interviewing also those who stayed will give you a fuller picture and better understanding of the underlying reasons for the churn.

Recruiting participants

The recruitment process can look different for different companies — from simply inviting previous customers to launching targeted email campaigns. Some companies even leverage Craigslist to recruit respondents, but it’s more for B2C businesses.

Being a B2B business, we need real customers for our research who use our product. That’s why we turn to our own user base to select respondents who fit the criteria. We also have a specialized landing page where visitors can fill in a Typeform and further be included in a user research group.

Once we define the research parameters, we send an email to those who match with an invite that has a direct link to an interviewer’s calendar in Calendly. This way, users can choose the day and time slot that fits their schedules. In addition, our support team sends an email directly to the participants offering to choose the date and time for the interview.

We use Airtable to keep the list of participants. But sometimes the list is too long and we need to narrow it down. In order to further qualify the candidates, we send a link to a Typeform with clarifying questions. Once we review the answers, we can make a more informed decision whether a person is the right fit for the interview or not. And if they are, they will see a link to the calendar at the end of the Typeform.

In our experience conducting interviews, users participate for different reasons. Some users love our product and appreciate what we do, so they want to help us get better. Others participate for incentives like an Amazon certificate or a pro plan subscription, among other things.

Preparing the questions

A successful interview is more than a simple Q&A session, it’s a conversation. So, it’s important to keep questions open-ended, also known as curiosity questions. This way, instead of a yes or no answer, a respondent is encouraged to dive deeper and tell you a story.

And these stories always take time — that’s why it’s simply impossible to fit a lot of questions into an interview. This is, actually, one of the most common mistakes of first-time interviewers who try to squeeze in as many questions as they can. Some make lists of up to 25 questions. But even if they manage to ask them all, they’ll have to do with some short and superficial answers, which is not the goal here.

A good rule of thumb is to prepare 8 to 10 questions, considering that almost everyone will have some clarifying or follow-up questions. That means that an interview is not a list of questions but rather a list of topics you can explore with your respondents.

However, there are a lot of pitfalls to avoid, like asking framing questions during an interview. Here are some tips to overcome them:

• Take the bias out. Questions must not be leading or in any way suggesting what you want to hear;
• Do not ask for compliments or take an aggressive position (i.e., asking “What’s wrong with that feature?” when a user is expressing their dissatisfaction);
• Keep the tone of your questions as neutral as possible but with a pinch of curiosity to encourage an interviewee to tell a story from their perspective;
• Actively invite people to share by asking a “How so” question (i.e., when a person tells you that they do certain things, follow it with a “How so” to learn even more)

We discuss more pitfalls and how to avoid them at our Power of Interview workshop.

Now it’s clear that you should always start with preparing a list of what you want to learn and only then move to writing the questions. If you do it the other way around, chances are you start fixating on the correct wording and see just the text in front of you and not the nuances.

Choosing an interview tool

At our company, we use Zoom, but you can use whatever tool is more familiar to your respondents. Some companies, for example, approve only specific solutions like Microsoft Teams. On a general note, make sure the solution you choose allows you to make recordings of your interviews.

Instructing respondents

Since our respondents are well-versed in using Zoom, we do not provide any additional instructions. However, that’s not always the case, and if you are not sure whether your interviewees are well familiar with the tool you are going to use, instructions are definitely welcome. 

Practice also shows that it’s worth reminding users about the upcoming interview. Some respondents forget and don’t show up despite the invite and the allocated time slot in their calendar. Others do not forget but tend to reschedule at the last moment.

To better plan your time, it’s good to ask users in advance if nothing has changed and, if their plans did change, offer them to reschedule. Calendly allows us to reschedule with ease — a respondent just follows a link from an invite and selects another time slot.

Assembling an interviewing team

An interview is generally conducted by an interviewer and a researcher who can also be an interviewer. You can also augment the team with an observer who performs the role of the second pilot, and if an interviewer becomes hesitant, an observer can step in. An observer can also be making notes, which is extremely helpful if the respondent did not provide permission to record the interview.

Since we use Zoom which allows us to record interviews, we have one interviewer conducting user interviews and several researchers who then watch and analyze the recordings.

Training before an interview

If you don’t have enough experience in conducting interviews, then it’s definitely worth having a trial run with your colleagues. A mock interview will help you see how you can better tune the questions to maximize the effect.

If you don’t have an opportunity to run an interview trial, sync with your team members after the interview to get feedback and adjust your approach. 

Screenshot from an interview session

During the interview

Here comes the most interesting part — talking to the participants and getting to know their stories.

Starting off the right foot

With so much going on in everyone’s lives, users can forget what the interview is all about. So always state the purpose of the interview and, if the interview includes sensitive topics, make sure to tell a respondent that they can always put the interview on pause or even stop it if they feel uncomfortable.

Then follow by describing the basic rules and asking the respondent’s permission to record the interview. Do not forget to mention that a respondent will have time for their own questions at the end of the interview.

Now it’s time to move to the interview. To make respondents feel more at ease, start with general questions. We usually ask about the respondent’s experience with customer journey mapping, their role, etc., and you can ask similar questions in the context of your product or services. Also, both interviewer and interviewee should have cameras to see each other during an interview. But if a respondent for any reason cannot turn the camera on or doesn’t have one, make sure your camera is on.

Stop worrying and help respondents do the same

Even the most experienced of us can feel nervous during an interview. If you do, the first step to overcome it is to acknowledge the feeling and be open about it with a respondent. This won’t take the stress away but will help make it more manageable.

Participants can feel nervous, too, so try to make them feel as comfortable as possible. If you see that a respondent is very anxious, pause and ask if the interview questions make them so uneasy.

Recording the interview

As we have mentioned already, we use Zoom which has recording functionality. But if a respondent does not agree for the interview to be recorded, you can take notes or ask an observer to do that for you.

One of the advantages of an online user interview is that a respondent is usually in a quiet place, which results in a high-quality recording.

More listening, less talking

Less experienced interviewers are very afraid of pauses and moments of silence. But you don’t have to because a pause can simply mean that respondents are just collecting their thoughts to answer the question. Do not rush to fill those silent moments as participants may just be thinking over the question at hand.

Asking again

Do not hesitate to ask again if something is not clear. Use a close-ended question to clarify what a respondent meant (e.g., I just want to make sure, did it happen last night?).

Closing on the right note

Keep an eye on the time. If the interview is supposed to take 40 minutes, then 40 minutes it is. And when you get closer to the end, you’ll see what topics have been covered and what not.

When you are running out of time, let a respondent finish the thought and then ask the last question while clearly stating so. This will give an interviewee a better understanding of the timeline. Once they are done, thank them for participation and move to answering their questions, if they have any.

Sometimes when respondents tell their stories, they can also give you valuable insights not directly related to the topic of the current interview. You may want to follow them up with some additional questions to learn more — ask for their permission to do so.

Genuinely thank respondents for their participation and if any compensation or a present is due, explain how and when they can get it. 

After the interview

The interview is over, but your job is not done yet. The post-interview stage is all about getting the feedback, analyzing the results, and tuning your strategy.

Performing a retrospective

Retrospective can be very helpful, especially if you don’t have enough experience yet. You can share the recording with someone more skilled and ask for their feedback regarding what other questions you could have asked. 

If it’s teamwork with several people conducting an interview, then the retrospective should be carried out in iterations after 1 or 2 interviews. Every discussion will help adjust the interview structure and tune the script to make it more effective (e.g., add more relevant questions or omit irrelevant ones).

If you conducted an interview by yourself, review the record after some time to correct any mistakes and work on your approach.

Following up

This is an obligatory step where we send an email to thank the respondents for participation and answer any questions that they had at the end of the interview, but we didn’t have time or couldn’t provide an answer at that moment.

Structuring information

We use Temi, an automatic transcription service, for transcribing interviews. If this is a short-term project, you can just sit and write down the main thesis.

But since we have long-term projects, it’s important for us to keep transcripts and properly structure them. We use Airtable for that. This way, we can always revisit transcripts whenever we need them and derive more valuable insights.

When working with transcripts, identify the main ideas and topics. If the tool you are working with allows you to use tags, structuring your notes can be even simpler.

Depending on the objective, you can use these insights for different purposes — from introducing new functionality to enhancing your custom journey map to identifying patterns for your personas.

The bottom line

On a final note, no online interview is possible without a reliable Internet connection. Take adequate measures of precaution like having a mobile modem or a smartphone ready to use a mobile hotspot.

As you can see, a lot goes into planning and conducting an effective remote user interview. But such meticulous preparation always pays off in the form of deep, actionable insights, better product design, and ultimately enhanced customer experiences.

Fortunately, there’s a growing body of academic research that illuminates exactly what makes defaults effective, including recent work by Eric Johnson, a Professor of Business and Decision Science at Columbia University. It turns out that there are three factors that drive the effectiveness of defaults. Understanding these factors is essential to determining if defaulting is the right approach for a particular touchpoint and, if it is, how a default can be amplified to change behavior or simplify a decision. Let’s explore the three factors in making a default effective:

Defaults help people avoid thinking about difficult things

The first reason defaults are effective is pretty clear: they reduce the interaction cost of making a selection. Defaulting saves the effort of reading, typing, or navigating to another option. As UX designers, we know people are always looking to save time and effort.

Beyond saving time and effort, it turns out defaults have extra power when we ask users to make a selection about something that is difficult to think about. Just like people tend to avoid doing things that are difficult, they also avoid thinking about difficult things. Here, too, there’s more nuance. What counts as difficult can come in a couple of forms:

1. A topic that is uncertain (i.e.: a calculation or an abstract topic)
2. A topic that is emotionally unpleasant (i.e.: a negative event)

An example of the first type of difficulty is when someone is being asked to decide what percentage of their paycheck, they want to set aside for their 401k or retirement account. It’s hard to know what the correct contribution level is since people don’t necessarily know how much saving each month will support them in the future. Accepting a default offers a way to avoid making those calculations.

Difficulty can also take the form of something that is simply unpleasant to think about. This is the reason defaults have been documented to be so effective in opting people into organ donations.

It’s not that deciding to be an organ donor requires complex calculations. Rather, it requires us to think about some unpleasant things. Namely, dying and what happens to your body after that. Applying defaults in settings like this enables people to forgo those thoughts and get the decision over with.

Tips for Designers:

1. Audit your experience for decisions, selections, or steps that are complex. Invest in finding a reasonable default for these steps and apply them.
2. Evaluate whether there are any decisions in your experience that require users to think about something that might be unpleasant. Find defaults that may work to help users avoid thinking about these things.
3. If you have a high degree of confidence that a certain option is best for users, increase the physical interaction cost of opting out of the default. This will increase the percentage of users that accept the default.

Defaults change the way users see their other options

Defaulting doesn’t just help people avoid thinking about hard things, it actually changes their preferences about the options available to them. For example, it’s easy to assume that most people come to every decision with a strong opinion about what’s best. The reality is that for most things, we make up our mind at the point when we are asked to decide. This is what psychologists call an “assembled preference”. What this means is that the value we attach to an option can change depending on how it’s presented.

Once someone sees an option defaulted, they actually establish a preference for that default. As unbelievable as it sounds, simply selecting an option increases its value in people’s minds. There are three well-researched cognitive biases behind this:

1. The Endowment Effect: the tendency to place a higher value on something if we have it in our possession than if we don’t.
2. Framing: the tendency to value the same options differently depending on which parts are highlighted in their description.
3. Anchoring: the tendency for a person’s estimate or valuation to gravitate towards the quantity they saw first.

When we default an option, people begin to think about the positive attributes of that first option. When they think about moving away from the default to another option, they’re primed to think about what they’ll lose by selecting something else.

Let’s return to our Retirement Savings Calculator example. There are pros and cons to contributing more or less aggressively to your 401k. If a person puts aside more of their paycheck each month, they’ll be able to retire earlier. If they save less, they’ll have more money to spend on stuff they want right now. Presenting a default choice influences the salience, or prominence of these benefits in a person’s mind, and impacts how aggressively they save for the future.

If the savings rate defaults to a level that allows users to retire at 59, it will call to mind the fun things they’ll be able to do while they’re relatively young. If they decrease their contribution, they’ll be more aware of the additional years they’ll have to work to retire. On the flipside, defaulting to a lower savings rate leaves more money in a user’s pocket each month. In this case, people will be more likely to feel the pinch of their paycheck shrinking each month as they increase their savings rate to retire earlier.

Tips for Designers

1. Consider pairing a default with messaging that emphasizes its benefits.
2. Increase the visual prominence of a default to ensure people see it selected first, and therefore, become anchored to it.
3. Consider offering a decoy option (one that is clearly inferior) in order to amplify the value of a default choice.

Defaults imply a recommendation

The third thing that makes defaults effective is that they can be perceived as an endorsement or recommendation of a particular choice. For example, if we default to a five percent contribution level in a retirement fund, users may assume this is an acceptable level. This Endorsement Effect is especially powerful if a person trusts the brand they are interacting with, and if they perceive the brand as knowing their needs and having their best interests in mind.

Tips for Designers

1. If there is strong reason to believe a user will be satisfied with a default, you can amplify the Endorsement Effect by making it clear the default is a recommendation.
2. Provide brief messaging around why a default has been selected and why a customer may want to stick with it. For example, let users know that a defaulted savings rate is agreed on as a good minimum by most financial advisors.
3. Look for opportunities to build credibility in your brand at the point your customer is being asked to make a decision. People are more likely to accept a default recommendation if they trust your brand.

Other factors that can influence the impact of defaults

Does each of these three effects ensure that defaults will always be effective? No. Actually, there’s one important factor that determines how likely a user is to accept a default: how experienced they are in making a decision or how strong their preferences are. If your users know what they need (and are invested in the decision) they’ll override the default. In these settings, imposing unnecessary defaults or making them too hard to adjust will create a frustrating experience. Unfortunately, that frustration will likely grow the more they use your product.

Defaults that are overly generic can also backfire. As we’ve discussed, defaults have a powerful influence on users’ selections. If a default succeeds in helping people move through a decision quickly, but doesn’t help them select something that is right for them, their long-term satisfaction with your product will suffer.

An approach that can help with this is to personalize defaults. Rather than applying one default across all customers, leverage what you know about your users to default to an option that is right for their particular needs. This could simply mean remembering what a user has selected the last time they interacted with your product. For products that aren’t used frequently, you can try to leverage data from a user’s profile or data that they’ve provided in other parts of your experience. For example, if you’re designing a benefits enrollment tool, you may want to default customers with young children into plans with more robust coverage since small children tend to need more medical care.

Summary

Understanding the psychology of how and why defaults are effective is critical to getting the most out of them. Research shows that defaults function in three ways: they eliminate the friction of having to think about difficult things, they anchor people to a certain value or option, and they can be perceived as a recommendation. However, defaults are most influential when they are personalized and when they are provided for people with less experience making a decision.

The overwhelming majority of the articles are “How to’s” and “What is’s”. I couldn’t find a single piece of content that extensively goes over the potential mistakes one can make.  Secondly, the only HE cons I’ve managed to dig up online have to do with the activities’ inherent flaws such as the inevitability of bias as opposed to the more practical, “real-world” problems and misconceptions. That’s what this article is going to touch on.

Before getting into the mistakes I see made consistently, let’s do a quick memory jog into heuristic evaluation 101. 

A Memory Jog

HE is the process of the evaluation of a product’s usability conducted by UX experts. Usually, it consists of the following steps:

• Outline the scope
• Start looking for experts
• Gather user intel
• Set up an evaluation system
• Analyze the findings
• Prioritize the issues
• Prioritize the issues

The set of heuristics used most commonly is Jakob Nielsen’s design heuristics.

The 10 heuristics are as follows:

The value of Heuristic Evaluation boils down to being able to identify up to 90% of usability issues relatively quickly and cheaply (as opposed to usability testing). In case you’re looking for a more comprehensive answer, here’s a video that covers everything in just 12 minutes . 

Also, here is an article I wrote a while ago on heuristic evaluation.

With the intro out of the way, let’s get into the mistakes.

The Most Common Heuristic Evaluation Mistakes

Heuristic Evaluation is not a substitute for Usability Testing

No matter whether you’re a world-class UX expert or just a novice, one can’t identify all potential usability issues single-handedly. Period. To make matters worse, not all ostensible UX issues are actual issues in the eyes of the end-user.

Don’t take my word for it. Here’s what Nielsen Norman Group, one of the world’s leading authorities on all-things-UX, has to say:

“Even the best UX designers can’t design a perfect – or even good enough – user experience without iterative design driven by observations of real users and of their interactions with the design” Source

Heuristic Evaluation by itself is divorced from the users’ input. As such, HE alone isn’t enough to eradicate all potential usability issues. The most efficient way to improve usability is to combine the two methods together. 

Involving just one designer

It is a truth universally acknowledged that a UX designer seeking to conduct heuristic evaluation must be in want of 5 users. The latter statement is barely news to anyone. When looking to stay on top of the latest news, start by checking this post about Jimmy John Shark. 

Starting from 6 and onwards, one starts observing diminishing returns. This is quite a well-known fact within the designer community. However, in an attempt to save resources, one will oftentimes find both clients and designers willing to save themselves the trouble of finding multiple designers.

Involving just one designer isn’t only a bad practice because one won’t identify as many issues as they otherwise would. Even worse, finding not the most experienced of designers can do more harm than good. Having multiple designers is also a moderately effective safeguard against a very opinionated or biased designer, who just likes things done his / her way.

Relying on internal designers only

As we’ve just learned, you’d need at least 5 designers to conduct Heuristic Evaluation. What hasn’t been discussed yet, however, is where do we find the UX talent. One could, of course, utilize the help of internal designers. That way there’s no need to search for talent elsewhere with all its associated expenses. Conducting HE internally does indeed sound tempting. However, I would advise against it.

In fact, I believe that having the design team who designed the app do HE on it is a terrible idea. It’s similar to having a local who lived in a city for twenty years give an objective opinion on how easy it is to navigate through said city. It’s simply not possible. 

That way, they won’t have any preconceived notions or biases. No matter how much you want to be objective, reviewing your own product is not the best of ideas. You get your designs, cause it’s a product of your mind. The latter doesn’t guarantee that an outsider will get it.

HE is less effective when dealing with a good design

Heuristic evaluation is based on design heuristics. No surprise there. However, unless the designer behind the product that’s being evaluated is clueless about the heuristics, it’s unlikely that a good piece of design will have blatant flaws. The more competent the design team, the harder it will be to find flaws, and the more data one would need to justify identifying something as “a flaw”. 

Additionally, the less apparent usability issues, the more a designer might wander into the territory of taste and subjective opinion. I, for instance, have a grudge against carousels. It doesn’t mean, however, that I should project my own grudge onto the users and advise to remove carousels to anyone who would listen.

Therefore, when dealing with designs that have no apparent flaws it would make more sense to do usability testing to improve the UX.

Heuristic Evaluation ≠ UX Audit

It is quite common, I would say, to hear “HE” and “UX Audit” used interchangeably. That has certainly been the case with my own clients. However, these two activities are different.

UX Audit, depending on the use case, can have multiple components that usually include:

• Heuristic evaluation
• Usability testing
• Desk research
• Quantitative analysis
• User Interviews

On the other hand, Heuristic Evaluation is a self-contained process that could be a part of a UX audit.

The distinction between the two is not a matter of technicality. Quite to the contrary. Calling heuristic evaluation a “UX Audit” gives both the client and the designer the illusion of comprehensiveness. The thought process goes as follows: “I’ve had my designer do the UX Audit (HE). Therefore, all my UX problems will be solved now”. Of course, this is not accurate.

Working with inconsistent heuristics

Even though the set of heuristics that is most common was developed by Jakob Nielsen, there are definitely more. To name a few there’s also:

• Gerhardt-Powals’ cognitive engineering principle
• Shneiderman’s Eight Golden Rules of Interface Design
• Weinschenk and Barker classification

This doesn’t mean, however, that mixing and matching the heuristics across multiple sets is a good idea.

The power of HE is the framework, i.e. the rules you follow to conduct a comprehensive audit. Cherry-picking the heuristics from multiple frameworks can result in a fragmented outcome. Additionally, the heuristics from different frameworks overlap which might result in doing a similar analysis multiple times.

Long story short: don’t mix and match among different heuristic frameworks unless you know what you’re doing.

Outro

If there were just one takeaway to get from this article, it should be this: heuristic evaluation is not a silver bullet. In isolation, especially when the proper procedure is compromised, HE can do more harm than good. Therefore, one shouldn’t just jump into conducting HE hoping it would solve every usability problem.

There are many ways to mess up a heuristic evaluation. Too many. I personally know that firsthand. However, I hope that some of the points made in this article will help avoid at least some of them. 

It’s a relationship world out there. We all know it, yet we treat our relationship with customers – or potential customers – as transactional in user research. As a researcher myself, I know that I need to get research projects done as soon as possible. There are so many phases of conducting a research study that recruiting and managing relationships seems too much.

As user research has scaled in organizations, the need for research operations —abbreviated as ResearchOps — became apparent. ResearchOps, as a discipline, emerged fairly recently to support researchers by establishing consistent research practices and processes needed to streamline work.

Through a dedicated ResearchOps team, researchers can focus on what they do best: conducting actual research. Without formalized structures, researchers can get bogged down in dealing with user research tools and research processes.

At the start of 2021, I switched from being a full-time user researcher to working on IBM‘s ResearchOps team. I still conduct research now and then, but my role shifted to building up our participant panels and participant recruitment. As I’ve been doing this work, our ResearchOps team sees another need arise: relationship operations or, as we call it, RelationshipOps.

In this article, I’m going to share the challenges I faced on my research operations team in the area of participant recruitment that led me and my team to see the need for relationship operations.

The ResearchOps challenge of recruiting participants

A key aspect of quality user research is talking to the right participants. After all, we pride ourselves on the user insights that we generate through our research methods.

However, recruiting participants can be challenging. In some cases, we may only need a handful of participants to answer our research questions, but we might need hundreds or even thousands of participants in other cases.

Outside of recruiting platforms, there are many different channels to find customers or participants part of the audience to work with. However, when you’re not using a dedicated recruiting platform, it can be challenging to convey the right value proposition to participate in UX research activities. I know I tend to ignore a lot of the surveys that come into my inbox!

To get the right people for studies, researchers often have to spend time identifying and reaching out to potential participants. Research teams also need to keep track of user consent forms and send reminder emails.

As a researcher on the research operations team, I can attest that these tasks tend to take away from a researcher’s primary goal: conducting user research. Hence, a ResearchOps practice around establishing streamlined recruiting practices and participant panels is necessary.

Legal challenge: Our research team can’t incentivize customers

At IBM, we have some serious legal issues around paying our customers. Any gift over $50, give or take, can be seen as bribery. The restriction makes sense in the sales context where someone from sales might butter someone up to purchase a product. However, we’re not looking to make a sale in user research. We want to have a conversation with them about their experience!

However, incentivization is a massive part of participant recruitment in user research practices. Offering someone actual money for an hour of their time is a solid motivator to participate in user research studies. We’ve tried various methods to incentivize potential participants without breaking any laws, but it’s been not easy.

Our ResearchOps team has a formidable challenge to support user researchers without incentivizing our customers. Our research team has to get creative in how we do participant recruiting.

Why does user research rely so heavily on incentives? It’s because user research, in general, is a transactional relationship. We have to get our work done, so participants are typically sought out to help us achieve project goals. This model works great when you can pay someone because you get their time and input while getting a nice incentive.

So we find ourselves in this situation where we need customers to talk to but can’t pay them. How do we get around this challenge? We have to set intentional strategies around meeting our customers’ needs so that they’ll be inspired to share their insights.

Finding the intrinsic motivations of customers

In user research, encouraging people to participate in studies presents a challenge. Customers want to talk with us because we have been doing good work for them, and they trust our expertise around user-centered design processes. However, if we continue to think of user research as transactional — i.e., “show me your product” or “tell me how you use it” — we will not be able to move beyond this hurdle.

User research needs to be more reciprocal; it’s important to move away from purely transactional relationships with our customers and start looking for the intrinsic motivations that will inspire them to participate in studies.

Intrinsic motivations could include a sense of curiosity or wanting to help out fellow users. It’s possible to find people who want to participate because they feel ownership over the product or service. There are many ways to tap into these intrinsic motivations — it’s just a matter of taking the time to find out what matters most to them.

Once we know their motivators, researchers can create engagement activities that align with those values, such as creating an innovation jam on a community forum. This way, not only do we get great insights from participants, but we also deepen our relationships.

Through RelationshipOps, we can give back to our customers

As user researchers, we’re constantly looking for ways to improve our research practices. We want the best people and insights to come out of user research activities — it’s what we live and breathe all day long! Yet as user research has scaled in organizations, there hasn’t been a lot of conversation about how user research teams should scale their operations. That’s why we have research operations, after all!

RelationshipOps is the practice of managing and nurturing relationships with customers or participants for the long term. It’s about establishing trust, rapport, and communication over time so that both parties can continue to benefit from the relationship.

Similar to Sales Operations or Marketing Operations, RelationshipOps focuses on the people side of things. It’s about building and managing processes to ensure that everyone involved can work effectively together.

Since our research ops team can’t pay for research with customers, we must build relationships.

I’ve seen user researchers struggle to understand how vital relationship-building is in user research. It’s very easy for user researchers who are used to recruiting research participants in user research studies to forget that they’re still working with people. I don’t mean that in a bad way. But just like we need to get user feedback, we also need to remember the participant’s perspectives.

Our customers use our products every day. The fact that they’re having trouble with a feature that isn’t on our roadmap doesn’t mean anything to them. They want their voice to be heard when they’re having a problem.

So, what about longer-term relationships with our customers? What if we could build stronger relationships with them?

Just as they don’t want us only to reach out when we need something, we need to be mindful of their needs and concerns.

So, what does RelationshipOps look like in practice?

There are a few essential practices that are necessary for RelationshipOps to be successful:

• Communication: Building trust and communication channels with customers or participants is critical for user research to be successful.
• Feedback: Gathering feedback and suggestions on what’s working and not working through user research activities helps everyone involved understand how to improve our practice over time — both the user researcher team and their customers or participants!
• Relationship Building: Building better relationships with our customers or participants is essential for user research teams to scale their operations.
• Participant Management: By establishing streamlined recruiting practices and participant management, research teams have appropriate knowledge management around the details of the relationships.

Final thoughts

In user research, the goal is to get a few people from our target audience or users of our product and talk with them about their experiences. We can either build relationships along the way or potentially never speak to someone again.

Through ResearchOps and RelationshipOps, the individual researchers can gather customer insights without learning another specialized area. The whole team meets customer needs.

Rachel Miles is a user researcher and UX Strategist on IBM’s ResearchOps team. The above article is personal and does not necessarily represent IBM’s positions, strategies or opinions.

There is no shortage of articles about how to ace an interview as a UX job candidate, but there has been little focus on how to be an effective interviewer. Assessing UX candidates can be particularly tricky because they can have highly diverse backgrounds, experiences, and skillsets. In this article, I will use several fictitious individuals to represent a variety of interviewer/interviewee situations and provide recommendations on how to avoid common interviewing pitfalls.

How to Establish a Good Repour with the Candidate

Fredricka is a senior user experience research manager at a large technology company and Malia is interviewing with her for a junior UX researcher role. After quickly rattling off her name, job title, and some boilerplate corporate FYIs, Fredricka jumps right in to ask Malia a series of intense questions to evaluate her experience with a broad range of research methods. From Fredricka’s perspective, she has a very limited amount of time to evaluate this candidate and she wants to make sure she covers everything. Her employer also has a list of behavioral questions that she is required to get answers to before the hour is up. Malia immediately feels intimidated and uncomfortable by Fredrick’s rapid line of questions on things that she has only had a bit of experience in her past internship and coursework.

Fredricka could have done a better job in this interview to make Malia feel more comfortable and to make the interview feel more like a conversation rather than an interrogation.

Have Empathy

We’ve all been interviewed for a job at some point in our careers and we know what it feels like. Unfortunately, having empathy tends to be the first thing to go when a person gets into interviewer mode.

The majority of candidates will be nervous or have some anxiety, whether or not it is visibly obvious while you are interacting with them. When people are nervous, it has been scientifically proven that they are more forgetful and their cognitive processes are impacted. This means that you aren’t interacting with this person at their best and their performance may not reflect how they would do on the job.

Things to consider:

• What might be going on in this person’s personal or work life?
• Could they be intimidated by your job title or the company that you represent?

Always take the mindset that you want this candidate to succeed. Create an optimal environment for the person to be able to present themselves in the best possible manner. You should be empathetic towards a candidate even if you ultimately decide that they aren’t a good fit for the job.

Be Human

Many interviews are now happening remotely and it is extremely tempting to multi-task. Just before the interview begins, make sure to close your email, set a do not disturb on your chat application, and put your cellphone on mute. Show that you are genuinely interested in talking to the candidate by frequently making eye contact (or looking into your webcam). It is ok to show emotion. Smiling and even laughing when appropriate can immediately put a candidate at ease and make the interview feel more like a friendly conversation. Sharing of thoughts and experiences shouldn’t only be one-way. Tell them a little about yourself. This could be a personal anecdote, something that you love about your job, or something fun about your team.

Have the perspective that this is not a test, but a conversation to determine a suitable match.

Be Curious

Conducting interviews can be a time-consuming process, especially when many of us are wearing many hats throughout the day. It can be convenient (or even advised by your company) to stick to a script of predetermined questions, but this can dampen our own curiosities to really get to know this person and their own unique story.

Calibrate your questions based on the individual’s level of experience. If the candidate’s only professional job related to this role is a prior internship, focus on what they learned during that experience. If the candidate has twenty years of experience, try to obtain a broader understanding of their background than just what they have done in their most recent job, if you need help achieving this, here are the 3 Best free reverse phone number lookup tools for background checks.

Find out what this person is most passionate about: Are they most excited about ethnographic research, design systems, accessibility, etc.? Their passion doesn’t have to align 100% with your own personal interests or the specific role that you are hiring for; Think about how they could bring more diversity to your team.

How to Evaluate Candidates

Sally is a senior UX designer working at a financial services company and Pablo is a candidate for a senior UX designer role on Sally’s team. Pablo appears nervous when thinking about how to respond to Sally’s question. Sally is thinking that she would have had a much quicker response to this question, and she is also starving, and can’t wait for her lunch break which is right after this interview. Sally isn’t really sure if Pablo knows his stuff compared with the person she just spoke to. That candidate works at a well-known start-up company, was much more confident in their abilities, and seemed like they could be a cool person to work with. At the very end of the interview, she realizes that she never gave Pablo a chance to ask any questions. She half-heartedly offers him a chance to ask her one question before she rushes off to lunch.

Unfortunately, Sally might not be evaluating Pablo in the most objective and open-minded manner and could miss out on hiring a valuable team member.

Remain Objective

Be aware of your own biases. Biases can easily hijack your decision-making process. They can control you—without you realizing it—and leave you wondering why you keep getting inconsistent results from your hiring process.

Don’t be susceptible to confirmation bias by making assumptions and prejudging someone before the interview: Just because someone has companies like Facebook, Google, or Amazon on their resume doesn’t automatically make them a rockstar. Or you might experience recency bias by comparing a previous candidate that you just finished interviewing with someone that you are currently speaking with.

The halo effect is a type of cognitive bias whereby our perception of someone is positively influenced by our opinions of that person’s other related traits. In contrast, the horn effect is a bias whereby one’s opinion of another is unduly shaped by a single negative trait. To combat this bias, force yourself to think about the candidate in a balanced manner. This can be difficult as we typically formulate an opinion of someone very quickly. If you find yourself reacting negatively to this individual, try to think about any positive attributes that you have observed so far. If you are reacting positively, try to think critically about any flaws that you noticed.

Remember, the goal is not to assess how good the person is at interviewing, but to determine if they are qualified and would be a good fit for the role. Just because someone provides “all the right answers” and appears extremely confident doesn’t necessarily predict how they will do on the job.

Probe: Ask questions about who, what, where, when, and why.

Your company may provide a set of behavioral questions that are recommended for you to ask. These questions are often phrased as “tell me about a time when you…”. Just because candidates are coached to answer these questions in a STAR structure doesn’t mean that it lets the interviewer off the hook to follow up. Interviewers often fail to probe deeper on these questions which can lead to a false or superficial understanding of the candidate’s experience or actual capabilities.

Here are a few situations where you should definitely take the time to follow up and investigate:

• There appear to be disconnects between their responses and their experience as presented on their resume.
• You disagree with what they said or think that the candidate provided a poor response.
• You weren’t confident that they understood your question.
• You weren’t completely sure that you understood their response.
• You feel like they didn’t completely or fully answer your question.

You should probe deeper to more fully understand the context of the situation, give the candidate an opportunity to provide clarification, and also for you to follow up with more specific questions about their response. Plan to ask at least a few follow up questions for each behavioral question that you plan to ask.

Don’t Waste Time

You likely won’t get to every question that you want to ask, and you shouldn’t attempt to squeeze them all in rapid-fire succession. Make sure to ask your most important questions first, give sufficient time for the candidate to provide an answer, and for you to follow up with probing questions.

You don’t need to ask the candidate for information that they have already provided on their resume such as which design or research tools they are proficient with or where they went to school. Don’t spend a lot of time telling the candidate things about the company that they could easily learn from visiting a website.

Limit the number of wide-open questions. Candidates have no idea how detailed a response that you are looking for and will typically overexplain. Instead, try to make most of your questions specific and give a sense of how in-depth they should be in their answer.

Also, keep in mind that questioning should be a two-way street. Do not wait until the last five minutes of the interview to allow the candidate to ask you questions. Try to answer their questions as openly and honestly as you can. Remember that they are also evaluating you as a representative of your company (and potential work colleague or boss).

Provide Feedback

Many interviewers are coached not to provide any feedback to candidates during the interview. This may be influenced by overzealous HR or legal teams that are afraid of interviewers saying “the wrong thing” which might have future theoretical legal consequences. You should be careful with the kind of feedback that you provide and thoughtful in how it is presented, but feedback should certainly be part of the interview process.

Providing constructive feedback is actually another way for you to assess the candidate. Observe how they react to your comments. It doesn’t mean that they must enthusiastically agree with what you have said, but do they appear to genuinely consider and embrace you providing feedback, or do they seem resistant, annoyed, or frustrated? The way that they react could you give you a sense of how they might respond to feedback if they were to join your organization.

Feedback shouldn’t always be negative. You can comment on why you liked a particular project in their portfolio or that you found a particular aspect of their background interesting. Positive comments throughout an interview can make the candidate feel more comfortable and often can help them to present a more genuine self.

Interviewer Checklist

During the interview, did you:

• Make the candidate feel comfortable
• Find opportunities to probe deeper to find out the what, why, when, and how
• Provide both positive and constructive feedback
• Give them ample opportunities to ask questions (before the end)
• Smile during the interview

Get Feedback on Your Interviewing Technique

It takes time and experience to improve your interviewing skills and it can be difficult to objectively evaluate your own performance. Make sure to get feedback on your interviews from either your boss or peers. If possible, record your interview sessions for them to review or allow them to sit in on a few of your interviews.

No matter the outcome of an interview session, the interviewee should always feel like they were genuinely listened to, had the opportunity to present themselves in the best possible manner, and that the interviewer was actively engaged in the conversation.

According to Visual Modo, more than 50% of consumers for goods from websites offering it in their language. Moreover, 72.1% of consumers spend all their time on websites in their language, 56.2% reported their ability to get information in their language is more important than the price of goods and services, according to the same source.

Many times, content is written in English in the user’s internet experience. But English makes up only 25.9% of internet users. You’ll see in this article how to localize your content so that it appeals to users locally and globally.

Content is king, but is it king internationally? You’ll find out in this article.

What Is Multilingual Content Marketing?

Multilingual means many languages, while content marketing is a marketing strategy used to attract and engage users through the promotion of content such as blog posts, articles, videos, podcasts, and more. Thus, multilingual content marketing is marketing content to consumers who speak different languages. Multilingual content marketing relies on the ability to be able to be relevant to your target audience–to know their language, culture, and trending topics, and to adjust your content accordingly.

Multilingual content marketing is a strategy of creating and sharing valuable content to attract and convert users globally. This goes beyond translating your content and indicates the process of content localization, which you’ll find out more about later, but for now, translating and localizing must go hand-in-hand for a creative multilingual content marketing strategy.

What Is The Concept of Content Localization?

Localization takes into account the linguistic aspects (including dialects and regional languages), and the cultural aspects (such as trending news and topics, and traditions and norms), to form a type of localized experience for a subset of a specific locale. Content localization means applying the process of localizing to content. It doesn’t really have a more specific definition than in-putting the regional culture and aspects of language into the translated text. So when I say content localization, I mean the process of localizing content, to serve companies that want to court consumers in their own country and globally in cross-border trade. In other terms, it’s a way to adapt specific content to your consumer’s needs.

In the article, Content Localization: When, Why and How Should You Localize Your Content they go over content localization and types of content you can localize. This blog post was from the website of Tomedes, which has that type of service and are experts in that field. They’re ranked as one of the top localization service providers by Global Brands Magazine.

Why Is Content Localization Important?

It’s not easy to directly translate your content and promotions without localizing it. By using direct translation, or even automatic translation, you’re only working with words–you’re not working to personalize your user’s entire experience. That’s where companies that offer translation services come in.

Localizing content is important primarily because it is a driving factor in multilingual content marketing. As I’ve said previously, localizing can personalize the experience of your consumers with your content, by making sure that your content is suitable for a significant number in that location.

What Is An Example of A Good Localized Content?

For example, usually, you don’t need to translate from US English to UK English as they’re both variants of the same language. However, if you really want to connect with your UK audience, you’ll localize it, which means in-putting the UK touch to the piece, like changing the word “bathroom” to “loo.”

Good localized content would mean bringing the locale’s touch to the table, so one good example for localized content is the airport signs that are translated. These airport signs are usually in two or more languages, and it’s a good “sign” if they’ve been localized too to reflect the expressions of the specific country/city.

There are many examples of good localized content from reigning companies. Here are two examples:

NETFLIX

Netflix has already had a slew of articles about its localization process, but the actual process of localizing a film still remains a secret, and we wouldn’t be surprised if they somewhat tweak the localization process for every film.

Nevertheless, Netflix has localized selections for all their locations, so if you’re in Poland, you’ll see different selections than if you were in Spain. They also combine their localized selections with personalization tactics that allow them to know what you’re watching now in order to suggest what you could watch in the future.

Here’s a photo of Netflix in Asia:

The biggest fuel to Netflix’s growth from a rented film platform to a full-on streaming service is its content. And it gives relevant video content to audiences globally through localization. It does this through dubbing, subtitling, and closed captioning to localize video content.

In terms of localization, Netflix mostly uses freelancers for their localization. Here’s a more in-depth guide to Netflix’s Localization Best Practices.

IKEA

IKEA has introduced IKEA’s Place, an augmented reality application that’s localized to visualize a real space with a superimposed IKEA product on it so that consumers can see what they want at their very own locations. The consumer can take a picture of where they’ll place the product, then choose from over 2000 products in the IKEA catalog. They then drag and drop the product over the photo, so they can reimagine how the IKEA product will fit in their space. That’s localization with personalization!

Furthermore, IKEA has also localized each of its pages for each location. For example, to match their newly opened store in Manila, they’ve localized their Philippines page, with the tagline: Quality Furniture for every Filipino home. They’ve also used Filipino models for all their images on this site. Meanwhile, their South Korean site, IKEA Korea, is also localized, with items for the Korean winters.

 As you can see, localized content can span a whole website, or take the form of VR and AR. However, a simple example of good localized content is a company brochure that has pictures relevant to the region of choice and translated text, since translating is a crucial part of contact localization.

How Does It Relate To User Experience?

Multilingual content marketing gives your business a competitive advantage in non-English speaking countries. Here’s why:

1. It builds trust with your audience – If you’re speaking their language, the consumer is more likely to trust your company than your competitors.
2. It leads to more conversions – Consumers are more likely to convert if you’re giving them the user experience that they need–through localized UI and visuals, through their own languages, and through their own localized experience. Consumers are more likely to buy from a non-English website, as research from CSA Advisory shows.
3. It shows a personalized experience – Having your website is not only good for multilingual UX, it’s also relevant for each individual user. If you’re connecting with each target user through a localized experience and a personalized one, you are more likely to be connected with your whole, global, audience.

Localization is important specifically for UX and web design as well because when personalizing the experience for the customer, such as dashboards, notifications, CTA buttons, and of course, content, then they must also be localized. Users in each market usually have unique preferences for browsing and check-out in websites, so the UX must also be localized.

Steps for a Multilingual Content Strategy

Based on what we’ve learned so far, a multilingual content strategy would need to use content localization. Here are some of the steps needing to be taken to take your content to the next multilingual level.

1. Understand your own core message – Understanding your own core message is the first step in any content strategy. In order to transpose it to others, you should have your own core message completely understood by you and relevant members of your company. Crafting your own core message for each region or each consumer takes much market research and overall planning.
2. Keep it simple – Keeping your core content simple will not be only good for your translated and localized content, but also good for your relatability with your consumers.  It must be clear, relevant, and easily understandable, in different languages, if you want to transform it in different languages. Best keep it free of idioms, over-the-top language, and other linguistic aspects that don’t translate well.
3. Create content for localization – Creating content for localization means taking account of your target audience and the feasibility of your own core message to apply to that target audience. Then, you create a copy that matches what you want to say to that audience with your core message within.
4. Localize and translate the message – Localization sometimes takes professional help–the help of professional localization experts. These experts will localize your message with Content Management Systems, AI tools, and more. Leaving it up to the localization team is a good tip to use.
5. Integrate International SEO – For all content on the world wide web, SEO, like the one serviced by SEO Freelance, matters. Once your content is made, localized, and translated, then you should not forget international SEO. International SEO is a conduit of Google worldwide, such as Google USA, Google UK, and Google España. These ccTLDs (if you don’t know what that means, brush up on your international SEO) should take up much of your keyword research on the specific regions.

Takeaways

Here are the three takeaways that I hope you get out of this article:

• Content marketing in online experiences must be provided in different languages, in order to make sure that the content is tailored to those audiences.
• Content localization is a necessary part of the process, making sure that the translated content is localized for the culture, the habits and the attributes of your local audience.
• International or multilingual content marketing starts local, but grows global, especially with SEO in the mix.

This summer, ORIGAMI surveyed over 200 companies in North America and Europe to understand their intranet usage. We found that only 1 in 5 intranets are used by about 90% of all employees daily; most intranets are used by less than 40%.

Any license and service costs companies spend on their intranet projects only deliver half in return!

To find out what companies with high intranet usage do differently, we looked at the results of our study for answers. We share the results of our findings to help UX professionals and strategists deliver better intranet adoption for their internal stakeholders and clients on an intranet project.

Here are the key attributes of successfully adopted intranets.

High Engagement Intranets are Data-Driven and are based on research

Our survey found that companies with high intranet engagement are twice as likely to collect metrics regularly, track them over time, and share them with leadership. In other words, the intranets are data-driven.

If your goal is to create an intranet that makes a real impact, we recommend tracking these user experience KPIs:

• Number of unique page views
  • This will tell you how many of your employees are visiting the intranet. More employees are visiting adds to higher intranet engagement.
• How deep down the page do you users scroll
  • This will tell you much more of your content was actually read. If your employees scroll further down the page, then they are more engaged with the content on that page.
• What are users searching and not finding
  • This shows what content users need and search for that is currently not present on your intranet. Optimizing content to user needs will increase intranet usefulness.
• What are users clicking on a given page
  • This will help you understand which links and calls to action are most useful. Using this information you can optimize the page and make it cleaner by having only the most important calls to action.

Some of the KPI measurements above are already available in many intranet platforms, and others can be added as extras (Google analytics, for example, offers some free tools).

Clean and Simple Design Attracts Users

One of the biggest worries leaders report in our survey is that their newly launched intranets might be hard to use, and employees won’t adopt them. From analyzing click heatmaps, we find that employees indeed value clear and simple access to their work resources, normally on the homepage. Also, in a qualitative survey, we found that, on average, 79% of employees say they appreciate easier navigation to the applications they use most frequently.

This sounds like a no-brainer.

So why is it so difficult for companies to make intranet pages clean and easy to use?

Unlike an external website, many intranets are too news-heavy. Communication managers use a homepage to promote all the latest news at once, losing sight of what most employees came there for—tools to get their work done.

This is when you see a homepage cluttered with article headlines competing for attention and overwhelming intranet users. What helps to reduce this information overload is targeting news to relevant employee segments. This is particularly relevant for organizations with geographically dispersed employees. With only relevant news on the page, you’ll have more room to promote quick links to essential applications that employees use daily.

Here are a few intranet examples and homepage design ideas that do this well because they combine news and access to work applications right at the front of the page without requiring users to scroll. This design principle applies to any intranet system you might be using.

Leading Intranets Give Employees What They Want by using inputs from the qualitative and quantitative study

When we surveyed employees about their top intranet requirements, 41% of respondents said they needed a way to find forms, templates, policies and procedures, and other work resources.

Employees often struggle with finding files buried somewhere on the intranet, and it can take multiple clicks to get to the right page. Many employees give up and ask their colleagues for help.

You can help avoid this by giving employees access to their work resources in a one-stop-shop experience. Instead of searching multiple pages, employees can find all the documents they need right on the homepage. Here is an example of how this can be done in a single application:

Interactive Features such as Polls, FAQs, and Videos can connect and engage

Employees are curious about what’s happening in the workplace, but much of the communication on traditional intranets is one-way and top-down. Often we meet UX professionals working with strict work cultures and reluctant to propose interactive tools and use of media. You don’t need to convince stakeholders to revamp the corporate culture to help bring some life into the intranet. You can start small by engaging employees in a relevant discussion using a poll or a video. We often work with customers with a very strict culture and find that a simple poll is always welcomed by employees and is likewise acceptable by managers. Videos and podcasts can be harder to produce, but they can bring huge engagement on many intranets where we’ve measured employee engagement.

Good UX requires a good understanding of UX by leaders

Last but not least, we found that when company leadership understands UX and actively supports the use of UX principles on an intranet project, the intranet is twice as likely to receive a highly positive quality rating from employees.

Why is that?

Businesses we surveyed with high leadership support reported that they have access to more resources to improve their intranet, more budget to manage and maintain the software. This results in more attention to employee needs and better technology and training, resulting in a better-quality digital workplace.

So how do you get leaders to become aware of the UX and its benefits?

• Become aware of what the company leadership is trying to achieve and demonstrate how UX can help achieve that
• Start with a smaller pilot project to demonstrate UX principles on a small scale

How one company improved their intranet employee engagement by using UX principles

Last year, we did a project for a company with 1,000 employees in high-tech manufacturing operating across three states. The company achieved significant improvement in employee engagement on their intranet by following some of the principles above.

This organization was migrating its on-premises SharePoint 2016 intranet to SharePoint Online. Early on, it became clear the company didn’t know how to measure the improvement they will gain from moving to a new intranet platform. The project lacked solid leadership support. The project team considered showing their new intranet page designs to a small team of 20 people to gather their impressions, but there were two problems.

• If new designs are visually appealing, does it also mean the pages are engaging?
• Second, how could the company maintain engagement with all 1,000 employees if they were not present during the review?

The issue was addressed in two steps.

First, new designs were reviewed with just the initial stakeholders to understand what content they needed on their new intranet. This helped inform design decisions and made the resulting designs more intuitive.

Then the designs were tested with randomly selected employees via an interactive survey asking them to find specific content on sample pages while their clicks were tracked.

The interactive survey only took about 5 minutes to complete per employee, and the results gave clear data about how employees engaged with the content. It was easy to adjust the design to be more intuitive and make information easier to find.

At this stage, the executive team already had measurable data on their new design. This was enough to give the leadership confidence to start building their intranet.

After the intranet site was built and launched, the company shifted into a second stage — measuring the effectiveness of key pages to sustain engagement.

Pageviews, as discussed earlier, can offer some insight into whether your intranet is used, but alone they don’t show you exactly where your employees click or engage. The company integrated an in-page analytics solution to gain a visual heatmap of where employees were clicking. This additional data helped this company see which parts of the page were not performing well or were too far down the page to notice. For example, we saw one trendy section of the page buried far below, and many employees had to scroll to find it. Of course, not everyone got that far, so engagement rates suffered (something we would not know why if we tracked only page views).

With that, the internal communications team could see which links need to be replaced and what parts of the page needed to change. In other words, the usage data provided internal communications with the tools to improve each month.

As a result, it was easier to report measurable improvements to the leadership, and, in turn, company leaders were more supportive and provided the communications team with much-needed support.

The Bottom Line

Each of these five tips can make a noticeable difference in employee usage of the intranet. You don’t have to use them all to be successful. Often even with a little bit of UX awareness, your stakeholders will be in a better position to help you drive UX principles on other company projects.

Rage clicks happen when users repeatedly click or tap in irritation on a particular element or area in a concentrated space of time. As users, we expect digital interfaces to respond to our actions in certain ways. When they don’t meet our expectations, we naturally feel annoyed. Rage clicking, and other forms of computer rage, are a physical manifestation of this feeling.

What do rage clicks signify?

Typically, if there’s a lot of rage clicking on a website, app, or another user interface, it’s a sign that something could be going wrong in the user experience. Possible causes include:

Misleading text or UI elements

Over the years, certain UI design choices have become more and more common. Some have reached the point of becoming established design conventions. Veering too far away from these conventions can cause confusion for users, who have become used to things working in a certain way. For example, underlined text is generally understood as a link, so users are likely to try clicking on it. Underlined text that leads nowhere could result in a mismatch between the user’s experience and expectations, triggering a rage click.

Slow responses and a lack of feedback

If your user clicks or taps an element and doesn’t receive feedback quickly, they may click repeatedly to confirm that the click actually worked. In these cases, users need to see a proper loading message or some other visual indicator. This will help them feel reassured that their action had an effect.

Errors, dead links, and broken elements

If a usually-interactive element becomes non-responsive users are likely to respond with annoyance. It’s important to ensure that the elements and links are up to date and functional.

Unclear headings or confusing navigation

Users want important content to be discoverable. They want to feel confident about what they’ll find when they click on a heading or a section on a navigation bar. Navigation should be as clear and unambiguous as possible, with a taxonomy that matches the user’s expectations. Vague navigation categories and unclear taxonomy could result in a rage-clicking danger zone.

Rage clicking is particularly likely to happen when the user is unable to see another way to reach their goal on the interface. If clicking doesn’t work, what are they supposed to do next? In these cases, you may need to make some UI adjustments to steer your users towards their goals in a more obvious way.

How Can You Identify When This is Happening?

You can observe how users are interacting with your site by using tools such as Hotjar’s user session recordings. They allow you to track movements like clicks, taps, and scrolls, revealing how users are interacting with your website. Some tools even have dedicated rage click detection features to help you identify this specific action.

A Hotjar session recording, showing how the user is moving across the page and where they have clicked.

What appears to be rage-clicking at first glance might not always be down to a user experience issue. It’s important to pay attention to context when you notice a clicking hotspot. In some cases, people will click repeatedly out of habit while reading or scanning a page. It doesn’t necessarily mean they’re frustrated or angry.

However, if you notice repetitive clicking consistently on certain elements or areas, and it’s coming from several different users, it’s time to take a deeper dive and identify the causes.

Some elements may invite repeated click events, such as zoom or volume buttons. In cases like this, repetitive clicks may not signify frustration, but simply users interacting with the page. It could be worthwhile to review these elements anyway to try and find a more efficient way for users to complete their task.

How We Turned Rage Clicks into Actions

Here at Cyber-Duck, we were able to use rage clicking as a signpost to help us make UX improvements for one of our clients, a housing-related organization.

When watching session recordings on Hotjar, our UX team noticed that the way users clicked around indicated that they were having problems finding what they needed. The important information was hard to find, sitting under confusing navigation with lots of options, or hidden in accordions. All in all, 339 different users engaged in rage clicking over a three-month period, with most of these clicks concentrated on a few particular areas of the site.

Eventually, many users gave up on finding the information themselves. They ended up going to the contact page, then making a phone call to the customer service team. Answering these inquiries took a huge amount of time and work for the internal teams. Meanwhile, customers were frustrated that they were unable to find information on their own.

Our team carried out extensive user research to identify how they could streamline and improve the user experience on the website.

We identified that one of our main goals was to help users to serve themselves when looking for the information they needed. To achieve this, we focused on four core principles: accessibility, flexibility, simplicity, and findability.

The user base spanned a wide range of ages and levels of computer literacy. It was important that the site would be intuitive and understandable for all, not just the computer-savvy. We wanted to make sure users were clearly signposted to the important information. If they felt empowered to help themselves, they could avoid the negative feelings that often lead to rage clicking. This would also reduce the risk of users simply picking up the phone, creating more work for the customer service team.

We carried out user research to uncover the reasons behind the rage clicks. One of the biggest pain points identified was that there were too many options in the original navigation bar and the taxonomy was unclear. The amount of information on offer was overwhelming the users, and they didn’t know where to click.

To resolve this, we wanted to segment users out at the beginning of their journey on the site by asking them to choose what type of customer they were. This meant they would only see content that was relevant for them. We designed a new navigation bar with fewer options and more sub-categories. This helped ease the user journey and point them towards the right content.

The simplified navigation bar sorts the user journeys into four key categories, directing users towards where they need to go. The key actions are then split into further audience types or actions. From these, users will then be directed to sub-domains or other pages on the site, which will have further navigation options.

Some other ideas generated were:

• Breaking up long content into more manageable chunks for improved readability
• Improving the search facility and offering related topics, giving users multiple paths to find the required content
• Providing clear calls to action to signpost users towards their goals

After the ideation stage, we created wireframes and prototypes incorporating these ideas. During usability testing, we received positive feedback on the clarity and ease of use present in the new designs. One user commented that the new designs were simple enough that even computer illiterate people would be able to find what they needed. In future iterations of the website, these UX updates will hopefully bring a much more streamlined, intuitive digital experience. We’re expecting many fewer rage clicks to be recorded as a result of these changes.

What Can We Learn from Rage Clicks?

It can be dismaying to see such a clear sign of a frustrated user on your website when you’re observing a session, but it provides a valuable learning opportunity. The nature of this action means it is often targeted on very concentrated areas of the interface. This means you’ll be able to zone in on very specific parts of your website and identify exactly what is causing frustration.

By looking a little deeper into the rage clicks, you’ll identify actionable steps to take to make the UI more clear, accessible, and intuitive. Even simple changes can make a huge impact on the quality of the customer experience. In turn, this can make a big difference to your conversion rates and business goals.