Focus on the user and all else will follow.
– Taken from Google’s philosophy ‘Ten things we know to be true’
Our design process began with gaining empathy for the user. Empathy is gained through carrying out primary research directly with the user. Gould, J. D., & Lewis, C. (1985) states that an understanding of the users is arrived at in part by directly studying their cognitive, behavioural, anthropometric and attitudinal characteristics. Because we were approaching this project we no prior experience or research done on Irish Rail we first needed to get a deep understanding of our user’s behaviour, frustrations and motivations for using the Irish Rail website.
While it’s been said that qualitative research is almost always the most effective means for gathering behavioural knowledge (Cooper et al., 2014) it is argued that quantitative research is key to gathering early stage data on a new user research project and for ensuring that it is fully understood who uses a particular product, how they use it and how often (Kuniavsky, M., Goodman, E., & Moed, A.,2012). Luke Wroblewski makes a brilliant point on the need of both quantitative and qualitative data in order to get the full picture. Using a Tour de France cycling race crash Wroblewski illustrates how different a feeling you get from looking at both quantitatively and qualitatively data types.
Slide from Luke Wroblewski’s talk @ Conversions@Google 2016 – https://www.youtube.com/watch?v=OkeJg92PA4E&t=4096s
The main reason for gathering data at all is to glean information about something (Preece, J., Rogers, Y., & Sharp, H. 2015). In order to know we we needed to research, we first needed to derive at research goals. Decided as a team, a list was made of what we get answered before we could define what we were designing and who for. Here are the main research goals we set out for the project:
- Identify any usability or user experience issues and any barriers / drivers to use.
- Understand current behaviour and experience of Irish Rail user
- Identify any usability issues when booking a train online.
- Evaluate the top user tasks of the Irish Rail website.
- Evaluate the efficiency of the Irish Rail website and measure it against user experience.
With our research goals were established we next focused choosing what research methods we would use to answer our questions. We looked to Rohrer’s 3 Dimensional Framework to help us decide the types of research methods were most suitable to answer our research goals.
Quantitative research is key to gathering early stage data on a new user research project and for ensuring that it is fully understood who uses a particular product, how they use it and how often (Kuniavsky, M., Goodman, E., & Moed, A.,2012). We used Survey Monkey to gather information about our users as it helped us to achieve our research goals of:
- Evaluating the top user tasks of the Irish Rail website and pain points.
- Understanding the behaviours of website users.
We began the process of creating the survey by collect the 10 main questions that we would like answered. We limited the survey to ten questions so that we could get a high completion rate and hence a high sample rate.
The survey contained questions are usage behaviour, frequency of using Irish Rail services, top tasks carried out on the Irish rail website and main issues that individuals have with the existing solution.
We received 54 responses in total from running the survey for 1 week.
We carried out comparative assessment as part of our desk research. We did this to help use to understand expectations of users when using a travel website and also to identify common patterns used across similar sites. For the purpose of our comparative assessment we focused on train, tram & airline websites.
You can view our comparative assessment document here:
We used card sorting to help us figure out the information architecture of the Irish Rail website (IA). Spencer D. (2009) suggests card sorting is great method for learning a lot about how people think about categories and concepts, how they describe them, and what information belongs to a category. [Cooper et al] argues while this can be a valuable tool to uncover one aspect of a user’s mental model, the technique assumes that the subject has refined organisational skills and that how he sorts a group of abstract topics will correlate to how he will end up wanting to use your product.
Given that we were particularly interested in learning about users’ top tasks and how the tasks related to one another we decided to use this method on a selection of our research subjects.
We used this method when interviewing our commuter participants. We did this be listing out the top tasks identified through our survey results and scattered them on a board as sticky notes.
We then asked our interviewees to sort the cards from top to bottom going from most important to least important to their needs.
One of our commuter users ordering the importance of tasks available on the current Irish Rail website.
From our surveys results were able to categorise the types of people use Irish Rail services and the main problems these users are having with the existing website. From data two prominent types of users could be identified the leisure traveller, and the commuter.
Since, one of our main research goals was to understand the current behaviour and experience of Irish Rail user we decided to interview both types of user to get a better understanding of how their goals and frustrations, and needs differed and if there we and overlap in the two that could present opportunities to solve problems for both while focusing on the one user type.
We carried our interviews with 6 participants in total.
We chosen to interview 6 Leisure users and 4 Commuters.
One of the best ways to understand people’s experiences is to see them for ourselves (Kuniavsky, 2012). Direct observation was carried out in a controlled environment on interview participants so that we could fully understand the issues that the users were having and to see how they carried out a specific task.
Combining qualitative methods
Given the scope of the project and the short timeline, it was decided upon to carry out observational tasks prior to interviewing users, so probing and clarifying questions could be asked about how users preformed certain tasks. In addition, Cooper et al. (2014), suggest that
combined interviewing and observation in a single session as is the most effective way of gathering qualitative. Finally, Goodman, K (2009) outlines how combining these methods will allow you to gather rich, useful information very quickly while minimising self-reporting error.
Note: It’s important to note that due to sickness in the first week of December I was only able to carry out a short observation one commuter participant. This observation was done in context with the participant check live train times for his evening commute home from work.
Alan & Rachel carried out the other observations on the leisure users. For this, the participants were asked to carry out a list of predefined tasks.
Preparing for the Interview & Observation
Now that we had our survey results, we decided to create a persona hypothesis that would help us to decide on suitable candidates by using screening them on these hypothesis. [Cooper et al] suggests that persona hypothesis should be based on likely behavior patterns and the factors that differentiate these patterns, not purely on demographics.
It is often the case with consumer products that demographics are used as screening criteria to select interview subjects. You can view the personal hypothesis for both our Leisure & Commuter below:
We used various channels to recruit our 6 research subjects. We recruited mainly through our social media channels and our survey respondents though some friends and colleagues were recruited as they met the screening criteria.
Preparing the script for Interview & Observation
We decided early on that our interview would consist mainly of open-ended questions but that closed questions would also be needed for the purpose of creating the persona. We did this through Google doc. I setup the doc and wrote the initial questions and the Rachel and Alan contributed to the questions.
You can view the full interview & observation script here:
An important bit of housekeeping before any research interview is to inform the research participant of how you will be using this research and get her official permission ( Buley, L.2013)
I created a consent form that our team asked each participant to give consent to the recording of the research session and to share the data among the team.
In the next blog post
I will outline how we analysed the data from our research study to define our persona and user scenarios. You can view that article here.
- Buley, L. (2013). The user experience team of one: a research and design survival guide. Brooklyn, NY: Rosenfeld Media.
- Gould, J. D., & Lewis, C. (1985). Designing for usability: key principles and what designers think. Communications of the ACM
- Preece, J., Rogers, Y., & Sharp, H. (2015). Interaction design: Beyond human-computer interaction (4th ed.). John Wiley & Sons
- Rohrer, C. (2014.). When to Use Which User-Experience Research Methods. Retrieved October 12, 2014, from https://www.nngroup.com/articles/which-ux-research-methods/