Measuring aesthetic and experience values using Big Data approaches
Led by: Prof Susanne Becken, GU
Project Summary
This proposed research responds to the urgent need of understanding how ecological changes affect the aesthetic value and the user experience of the Great Barrier Reef, and how these could be measured and monitored in a cost-effective way. The research capitalises on two major trends, namely peoples’ ability and willingness to share large amounts of information through various online platforms, and rapid development in computing technology to store, process and interpret these data.
Project Publications
Project Description
Research Problem
The Australian Government is investing considerable resources into improving reef health and monitoring changes of the Great Barrier Reef (GBR). Monitoring however, is costly and the notion of citizen science is attracting attention. Involving people and understanding the human dimensions of the Reef, including natural beauty as an integral part of heritage value, is critical in enhancing support for conservation. Tapping into user-supplied material, for example through social media, reflects an anthropocentric approach that accepts that both aesthetic and experiential values are concepts that arise from the interactions between nature and people. Building on existing citizen science programs (e.g. Great Barrier Reef Marine Park Authority’s (GBRMPA) ‘Eye on the Reef’), this research draws on information shared by over 2.7 million people who visit the GBR (GBRPMA, 2018) to extract insights into environmental and experiential attributes of the GBR.
The aim of this project therefore, is to integrate work on understanding, measuring and mapping aesthetic value of the GBR, evident in user-supplied imagery, with research on measuring experience value, expressed through visitors’ sentiment and emotions contained in social media. Whilst it is likely that changes to the aesthetic value affect visitor experience and sentiment, it is important to capture these constructs separately. The reason for this is that, at least for some user groups, visitors to the Reef continue to have enjoyable experiences, even in the face of ongoing declines in Reef health and aesthetic value.
Project outcomes to date
Two previously funded projects (2.3.2 and 3.2.3) used Big Data approaches to assess how unconventional data sources (e.g. Twitter, Facebook, Flickr) can help monitor environmental and aesthetic conditions of the Reef. Using language processing tools (sentiment analysis), data visualisation (e.g. heat maps), deep learning (automated species identification), eye tracking (aesthetic rating), and neural networks/AI (automated beauty scoring of photos), these two projects delivered an innovative basis for enhancing current management systems of the Great Barrier Reef. Findings were shared with a range of key stakeholders, who expressed keen interest in further advancing this line of research. A list of outputs is available on https://www.griffith.edu.au/institute-tourism/our-research/sustainable-tourism-innovation/monitoring-the-great-barrier-reef.
This extension project focuses on understanding and measuring two key concepts relevant for ongoing Reef protection, namely the (underwater) aesthetic value and the experience value of the Reef. Assessment of changes in ecological value are out of scope.
Most data required for this project are open source and online. Recent findings by this research team found, for example, that Twitter users posted a total of 700,782 tweets from the GBR region for the period from March 2016 to March 2018. Of these, 163,691 tweets related to key aspects of the Reef (e.g. beach, fish, coral). In 2016, we downloaded 6,390 Flickr images that were tagged ‘GBR’. Of these, 1,440 images carried detailed geographic metadata. In addition, the extension research project will generate new data, for example from continuing eye-tracking experiments building on earlier research. We also collaborate closely with the CSIRO on their collection of new data from those involved in Reef-related activities (e.g. tourism, citizen science) and will ensure use of these data as part of the Big Data platform.
1. Aesthetic value: Further improvements are required in three areas.
- What are the attributes of aesthetic value? To verify and extend earlier findings (NESP 3.2.3 and 3.2.4) the relevance of additional attributes need to be tested, in particular those that could detract from aesthetic value (e.g. man-made structures). Further, the robustness of aesthetic attributes across different user groups needs to be verified. In particular, we propose to use eye-tracking technology in combination with interviews for two key groups, namely Indigenous peoples and Chinese tourists, to further understand perceptions of ‘beauty’ and relevance of particular attributes (e.g. charismatic species).
- To enhance and improve the accuracy of automated aesthetic scoring, using neural networks/AI, more ‘training material’ in the form of rated images is required. In particular, more images that show ‘ugly’ or less attractive underwater scenes are necessary to inform the machine learning. The eye-tracking findings will inform the selection of additional material. We also plan to use material previously rated in a CSIRO survey (from NESP TWQ Hub Project 3.2.4) and manually rated images generated through the current CSIRO proposal (5.6 in RPv5).
- The process of accessing and storing large amounts of images (e.g. geo-tagged Flickr) needs to be improved in its efficiency. In addition, buy-in from stakeholder groups that are already involved in various citizen science projects would be beneficial to enhance the flow of incoming images that can be used for aesthetic rating. Dedicated workshops will help increase support and link systems. Here, we will collaborate with CSIRO.
2. Experience value: Whilst considerable progress has been made in terms of capturing and analysing social media data, further steps are required.
- To improve the accuracy of the sentiment algorithm, we need to generate data from human annotation (i.e. coding) regarding sentiment and targets (i.e. what is the social media text about). A small number of relevant targets need to be agreed on with stakeholders, for example ‘marine life’, ‘coral’, ‘beach environment’, ‘water related’, and ‘tourist activity’. The annotated text will then be used for machine learning to improve the accuracy of the sentiment algorithm and also to produce target-specific sentiment.
- Further work is necessary to disaggregate sentiment from social media into categories that are of greatest relevance to stakeholders, for example by segmenting data into geographic zones, targets (see above), and user groups (e.g. domestic, international, local). This process will also involve consideration of other social media sources, including Sino Weibo data.
3. System trial: The proposed measures of aesthetic and experiential value will be tested on a specific intervention. More specifically, and due to recent Government investment and interest, the practice of coral restoration will be assessed.
- Perceptions/ Experiences: Sentiment of social media text related to ‘coral restoration’, expressed in various online media, will be compared with data that has information obtained in interviews with visitors who experienced coral restoration projects.
- Aesthetics: Restored coral will be tested by using relevant imagery in a dedicated eye-tracking experiment to assess people’s interest/attention (e.g. fixation time), beauty rating, and emotions (e.g. facial expression and self report). The images of restored coral will also be tested with the AI automated system to compare beauty scores.
Prior research and link to other research
This project builds on NESP-funded work through projects 2.3.2, 3.2.3 and 3.2.4. It also links to project 3.2.2. on the Human Dimensions outcomes, objectives and targets in the Reef 2050 Long-Term Sustainability Plan, specifically the socio-economic and heritage monitoring.
This project will benefit from the TrISMA (Tracking Infrastructure for Social Media Analysis), a joint project by several Australian universities and led by the QUT Digital Observatory. TrISMA will facilitate cost-effective access to large volumes of Australian social media data that will complement data from international users already streamed and stored at Griffith’s Big Data lab.
Synergies with several citizen-science approaches through the NESP Clean Air and Urban Landscapes Hub are possible (https://www.nespurban.edu.au/participate/). Also, there could be a future link with Project 6.5 (Citizen science) in the Threatened Species Recovery Hub.
Informing decision making
The understanding of aesthetic value will help Reef managers and those working in the tourism industry in their development of educational and promotional material (including for Reef protection campaigns). Already, the 3.2.3 research found that certain attributes (e.g. vividly coloured fish) significantly add to perceived beauty of the Reef.
The AI-based system will deliver a mechanism for monitoring aesthetic value at a large geographic scale, as it responds to images posted online or added manually to the database. The development of a final tool, for example a real-time map that visualises beauty scores, can be undertaken and implemented as part of the Reef Integrated Monitoring and Reporting Program (RIMReP) program. We suggest that the combination of manual monitoring at selected indicator sites (CSIRO) with automated monitoring – that is also constantly improved through the use of new training material (i.e. the annotated photos supplied by CSIRO) – provides a robust and cost-effective mechanism.
Similarly, the ability to measure and monitor sentiment (aggregate and for user-defined segments) provides intelligence on how users perceive the Reef, what they talk about, and whether there are any particular issues. Measuring sentiment in addition to aesthetic value is of critical importance to operators and related communities who rely on people having a ‘good experience’ as a summary indicator of destination success. The fact that the proposed system stores all Reef-relevant tweets in a database represents added value as data are available for any additional extraordinary analyses. Already, we have had interest in the dataset by a range of Reef stakeholders, for example in relation to the coral bleaching evens in 2016/17.
NESP 2017 Research Priority Alignment
This proposed research aligns with several of the NESP priorities and cross-cutting issues. First of all, Priority 1 on climate risk is explicitly taken into consideration by the systematic testing of different levels of ecosystem health (including in particular coral) on aesthetic and experience values. Climate change impacts and water quality issues are the two drivers that affect the attributes that have already been recognised as being integral to these Reef values and our research will she further light on the relative importance of attributes across diverse user groups. Priority 2 specifically addresses social and economic values and these are at the heart of our research. The research will be particularly relevant to the tourism industry and all communities that depend on it.
The research supports several NESP TWQ hub priorities. At its core, understanding and measuring aesthetic and experience values links to the notion of citizen science and involving people in the process of generating data and increasing awareness (3.6). It also tests the impacts of a specific management intervention, namely coral restoration (3.1), on socio-economic values (3.4). Finally, it helps increase understanding of how some of the cumulative pressures that manifest in declining Reef health value impact aesthetic and experience values – information that is important to design effective management of the asset (1.4).
Project Keywords
Aesthetic value; Experience value; Big Data; Social media; Artificial intelligence.
Project Funding
This project is jointly funded through GU and the Australian Government’s National Environmental Science Program.
