Name of Intervention/ Program
Improving Effectiveness of Public Transportation Communication Messages and Materials Using Neuromarketing
Background and Situation Analysis
Testing communication materials prior to large scale dissemination is a vital step in campaign development by ensuring that messaging is responsive, appealing, relevant, and persuasive for the priority audience. When investigating studies within the transportation sector, reference to concept-testing, pretesting of messages and materials is scarce or lacking; as the emphasis is often on the implementation and evaluation of communication efforts. We found efforts that have been dedicated to guide message placement, where data-driven marketing techniques such as market size analysis, data analysis of geocoded national data sets, service quality analysis using cluster analysis and decision trees to know how and where to market. However, often little effort is focused on testing the messages with the priority audience prior to wide-scale dissemination. In fact, though there have been calls for increased attention to emerging technologies around message dissemination and testing, we found a lack of studies describing the testing and revising of communication materials by transportation agencies prior to release. Pretesting has great potential to improve the effectiveness of printed messages and applicability to commuters (the priority audience). Methods used to test messages have traditionally included surveys, individual interviews, and focus groups. While systematic and valuable, these techniques only measure what the consumer is aware of, what they are willing or comfortable in sharing, and more importantly, how they wish to be perceived. Therefore, there is a level of bias in how people self-report their review of communication materials, such as advertisements, websites, apps or public service announcements. Neuromarketing is an innovative and useful tool in combatting biases in the system because it connects on a personal level with the raw biological responses of the consumer/audience. These techniques have come to fruition and have been used by social behavioral researchers to assess consumer attentiveness, emotional response, memory, and intention to change. These techniques use eye tracking, face reading technologies and skin conductance, to assess attention, emotion, and intention to change. Ultimately, the understandings gained from this technique help commuter assistance programs (CAPs) and transit agencies’ marketing programs to strengthen engagements with their priority audience of commuters and riders to create more effective marketing with results that ultimately impact behavior to deliver social benefits such as reduced emissions and decreased traffic congestion.
Priority Audiences(s)
CAPs and transit agency professionals offer transportation alternatives to commuters for the purpose of relieving traffic congestion and improving air quality. They advocate for more use of carpooling, vanpooling, teleworking, bicycling, and riding transit. They use a variety of marketing techniques such as print brochures, apps, websites, financial incentives, and community outreach events to reach priority audience of commuters.
Behavioral Objectives
This study sought to apply neuromarketing techniques to ascertain the effectiveness of a variety of existing CAP communication materials and re-evaluate the effectiveness after revisions to the materials were made based on insights gleaned from neuromarketing analysis. We hypothesized that the use of eye tracking and eye-gaze mapping, coupled with traditional market research techniques could help improve the effectiveness of the communication materials. Specific objectives include: • Increase the likelihood that the modified communication material would induce action. • Identify which aesthetic elements compel participants to focus on the material. • Increase the relevance of the materials to the priority audience.
Description of Strategy/Intervention
Through the use of eye-tracking, eye-gaze mapping, and market research, this study sought to evaluate existing communication materials and understand commuter’s subconscious engagement and response. Using a community-engaged approach, we narrowed a field of 121 materials down to 15 with seven distinct calls to actions. The materials were tested, revised and retested using neuromarketing techniques. The effectiveness of the materials increased with regard to clarity of call to action, relevancy, ability to persuade and intention to change. This case study illustrates the process used, the results and recommendations, which include – using simple and clear messages, taking advantage of classic gaze patterns, improving flow through formatting, and placing logos and QR codes away from messages. This paper illustrates the importance of testing messages prior to large-scale dissemination to improve effectiveness and to use techniques that counter response bias.
Implementation
This study included a multi-phase protocol. Phase I, included stakeholder engagement and identification of CAP communication materials from around the United States. Results from this phase were used for Phase II, which included the testing of the selected communication materials via eye tracking. Phase III, involved the revision of communication materials based on Phase II results, and retesting materials with a new sample. Note: We will use one of the posters below to illustrate the process. Phase I We solicited the collection of communication materials from around the United States from various CAPs. Using the Transportation Demand Management (Transp-TDM) listserv, as well as discussion with local CAP district leads, we collected a total of 121 communications materials from around the US, which were categorized according to the seven “call to action” or the what the commuter was supposed to do after having reviewed the communication material. The seven calls to actions were: register/enroll/attend, change of mode, contact/call/visit us online, download the app, form/join, log/track, and redeem. Materials consisted of social media ads, website front pages, public service announcements and print/digital communication materials. The stakeholders were asked to review the communication materials categorized according to call to action and select ones that reflected clearly the call to action. This resulted in the selection of 12 print items and three (3) videos. These materials were then used for the following phase. Phase II The communication materials were randomly assigned to three groups. Each group consisted of four (4) print items and one (1) video. The four print items were presented randomly within each test group first, and the video public service announcement was presented last in each group. Participants advanced to the next piece of print material on their own initiative and had their eye position collected while viewing the print and video public service announcements. Testing used iMotions software to record and analyze ‘eye tracking’ to answer questions pertaining to what subjects spend time looking at in the stimuli and identify particular ‘areas of interest’ for further analysis. Study participants answered a series of survey questions before viewing the material and after viewing each piece of promotional material. The respondent survey questions addressed demographics and evaluated the perceived effectiveness of materials in persuading commuters to change their behavior (follow through with the call to action/main message). The following domains were used to define perceived effectiveness: • To what extent do participants understand what they are being asked to do (comprehension)? • To what extent do participants find the ad acceptable or applicable to them (acceptability)? • To what extent are the materials attractive to participants and why (attractiveness)? • How relevant are the ads to the participants and their lives (relevancy)? • How compelling are the ads in engaging participants to complete the call to action (persuasiveness)? Data Analysis A combination of heat maps and area of interest (AOI) fixation data were used to analyze the eye tracking data from Phase II. While the software provides many eye tracking data variables and visualizations, the data considered in the analysis was Time to First Fixation (TTFF) and Dwell Time from the AOIs and fixation heat map visualizations. The aggregated Dwell Time is a representation of the order material was viewed and amount of time associated with each AOI was reported as a Dwell Time. The Heat Maps presented a visual representation of eye fixations with red, yellow, green, and no color representing varying levels of fixations overlaid on the collateral. The survey data was downloaded from the software and analyzed using Excel. The eye tracking information in combination with the survey data were used to select the four (4) promotional items to be modified prior to retesting in Phase III. Phase III Four (4) print items were selected for Phase III of this case study. The four (4) items were selected based on potential for improvement and affiliated CAPs program partners. Three (3) of the print items showed high potential for improvement and one (1) piece of collateral presented significantly less opportunity. Using the survey data, heat maps, and AOIs we made modifications to each piece of collateral. The changes included relocating material, updating icons, rewording, and alternative placement and sizing of visual elements. In each case we sought to stay consistent with the color scheme and message intent of the material. Next, the modified four (4) print items and same survey questions from Phase II were tested with a new convenience sample of demographically diverse students sourced this time from a graduate student workshop pool. Phase II testing process and analysis were repeated for Phase III.
Evaluation Methods and Results
The following will summarize one of the four materials that went through all three phases. The communication material showed two women talking while holding coffee. The collateral has the caption of “Let’s Make Oregon Better.” The verbiage is concentrated on the left side and the women occupy the entire right side of the page. The call to action (CTA) was ambiguous with a date for a challenge with no additional context. As shown on the attached visual, the first-round pretesting eye fixation heat map revealed numerous “hot spots” on the left side verbiage along with the women’s faces and right woman’s coffee cup. AOI fixation data indicated participants first viewed the “Let’s make Oregon even better” and spent little on the detailed information but instead, jumped to the woman’s faces. Compared to other elements in the collateral, the participants spent a relatively long time on the left women’s face (second longest dwell time). Despite providing no message content, the “Get there” logo on the bottom left occupied significant real estate and produced a fixation hot spot. The “Get there” logo also roughly duplicates the “Get There Challenge” verbiage at the top of the text column. Not surprisingly, the survey results showed that this collateral was not effective. Nine (9) out of 21 participants thought the collateral was confusing, and only four (4) people identified the “Rewards” message in the ad. Revision of “Let’s Make Oregon Even Better” for Phase III Testing In hopes of improving the collateral effectiveness, we took advantage of people’s interest in the faces and the lack of content on the right side of the collateral by delivering a CTA action between the two women as if they were speaking to each other about the CTA. “Win Prizes” and the data were located between the faces. A second major modification was shrinking the logo and relocating the logo to the to the bottom right to not compete with the CTA. Next, to improve clarity, “Drive less” and “Use Transportation Options” were added to illustrate how to make Oregon better. Instead of repeatedly mentioning “Get there,” we added a CTA section, which included detailed steps on “how to win prizes”. Just two steps were displayed at the bottom left to show simple actions to unlock the rewards. Finally, we placed a QR code next to the unlock steps so that participants could access detailed information to participate in the event. Phase III Testing Results TTFF results from the Phase III retest of Let’s Make Oregon Better, showed the top left corner was the first place to be noticed, and from there, respondents’ eyes shifted to the bullet points and the unlock steps below. Based on the fixation heatmap, the “Win Prize” located between the two faces drew the fixations away from the faces and the coffee cups. The survey results indicated respondents better understood this modified collateral. Fifty-six (56%) of respondents identified “Redeem Rewards,” as the key message in the collateral compared to 5% in the Phase II test. “Form/Join a Carpool” and “Download App” also increased from 24% to 69% and from 24% to 63% respectively in Phase III. Regarding acceptability, 63% and 38% of the viewers could identify the benefits described in the collateral, increasing from 19% and 5%. Moreover, 44% of 6 participants believed that they would be somewhat likely to act because of the attractiveness of the collateral, and 50% believed they would somewhat likely be persuaded. In contrast, 43% of participants thought the original collateral was confusing and only 19% of the respondents thought the Phase III collateral was confusing. Lastly, participants also reported the revised collateral was more relevant to their lives or those around them, with 88% of participants saying they thought the collateral was for them (35% increase from Phase II testing) and the percentage of respondents who chose “No one I know” decreased from 48% in Phase II to 6% in the Phase III retest.
Entry Letter: LL