Evaluation studies are typically undertaken in accordance with a Methodology Handbook (MH) that is devised during the project initiation phase. The following table presents an overview of the methods that were deployed in a phased approach during a five (5) month period, from July to November 2017 for the evaluation of three key website services.
The project started with an initial content audit that took place to allow the Service Team members to familiarise with the website's content, the CMS and the objectives of the study. Following this onboarding phase, the evaluation methods were rolled out gradually to web audiences. The majority of the outcomes of the methods were available towards the end of October 2017, when the Service Team undertook the synthesis of the recommendations and the compilation of design recommendations.
For each of the methods used in the project an initial analysis took place in collaboration with the Client in order to define the instrumentation of the study, the planning of its roll-out and key promotion activities to recruit participants. Following the initial analysis, the data collection phase aimed to deliver the studies (collect data, perform user sessions, etc.). Finally the reporting phase compiled the results of each method in a stand-alone report presenting the study’s detailed findings.
In the context of the given assignment a set of fit-for-purpose behavioural and attitudinal research methods were used to assess and improve the usability and user experience of an existing website.
An Expert review pertained the heuristic evaluation of the user interface for compliance to well-defined criteria (heuristics) by at least two usability experts. Findings were later used to improve the usability of the given website.
SEO and Web Analytics Audit
A structured inspection and assessment of the website in question aimed to describe the following business elements:
- The website's communication and operational objectives: aiming to acquire domain-specific knowledge and identify the information and communication requirements that the website fulfils.
- Web traffic analytics: aiming to understand the website's traffic patterns in terms of the web audiences' interactions with the website (e.g. pageviews, sessions, returning/new users, engagement levels, and geographical locations), the content indicators (e.g. top content, landing pages, exit pages, users flow, and site search terms) and the channels/referral sources to the website.
- The website's speed and performance aiming to ascertain if the web technologies used are optimised for web and mobile devices and gain insights on how to optimise the pages' critical rendering paths and page weights (for selected pages).
- The website's search engine performance in orderto understand how effectively the website is indexed and crawled by search engines, how the key website pages rank in search engine result pages (SERPs), what are the top search keywords, and document the identified on-page optimisation actions (keyword placement and relevance to search queries) especially for pages with low click-through-rate and/or low dwell-time (high bounce rate with low time on page).
Online User Surveys
User Surveys were utilized to gather data concerning the actual usage of the website and the available services and features. The main objectives were the following:
- Assess the use of the available functionality;
- Assess the users' view with regard to the navigation and information architecture;
- Identify levels of satisfaction with various aspects of the user interface;
- Assess the user needs for additional products and functionalities; and
- Provide benchmarks of usage and usability indicators for comparison in future surveys.
User Interviews and Focus Groups
In the chosen methodological framework, user interviews provided direct contact with external users and agency stakeholders and staff, aiming to describe who they are and what are the main tasks they try to accomplish as well as their concerns and needs. User interviews were semi-structured and were carried out remotely and through planned site visits.
Focus groups were deployed (a) in order to engage multiple users in thought-shower sessions especially for new/emerging needs and (b) when complex requirements needed to be discussed or requirement conflicts between project stakeholders needed to be resolved.
Personas and User Journeys
Personas are realistic representations (profiles) of the key web audience segments of the website and are used to describe and communicate the users' expectations of the website.
Personas were built through a review of web analytic data, user survey responses and primarily through direct user interviews. Personas were used as a validation tool in the design stages of the project (supporting the evaluation of new requirements, layout, targeted campaigns, labelling and informing the website's content/communication strategy). Depending on the complexity of the assignment the personas can be extended through the definition/mapping of the most important actual user journeys ( User Journey Mapping ) to uncover possibly problematic user journeys and define the conversion funnels. User Journey Mapping was deployed to assess the effectiveness of complex hierarchically-structured websites and multi-step conversion funnels.
Low and High Fidelity Prototyping was employed as a technique to quickly visualise new ideas, alternative concepts and review the usability and stylistic approach to addressing identified usability issues and elicited user experience insights.
The effectiveness of the website's actual sitemap was validated with IA Validation software in order to answer structure and labeling-related questions and address findability and discoverability issues. The results lead to the creation of an improved sitemap in the form of a flow diagram that identified all pages and showed the pathways linking each page.
Usability Testing involved the design and implementation of formal usability experiments that aimed to collect suitable qualitative and quantitative data in order to uncover important issues relating to the usability and accessibility of given user scenarios. Usability test sessions were organised with a fixed number of users (approximately 10) who were asked to perform a number of typical test scenarios or tasks. The Usability Testing approach that is typically employeed can be moderated or un-moderated, performed in a usability test lab or remotely through the participant's computer.
The objective of usability testing was to derive actionable recommendations related to the usability criteria of learnability, efficiency, memorability, errors and subjective user satisfaction.
Many thanks to Eworx for giving me the opportunity to be a part of a team of unique UX practitioners and most of all amazing individuals.