Articles | Volume 6, issue 3
https://doi.org/10.5194/gc-6-97-2023
© Author(s) 2023. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
https://doi.org/10.5194/gc-6-97-2023
© Author(s) 2023. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Research article
| 
06 Sep 2023
Research article |
| 06 Sep 2023
| 06 Sep 2023
Understanding representations of uncertainty, an eye-tracking study – Part 1: The effect of anchoring
Kelsey J. Mulder
CORRESPONDING AUTHOR
Brian Hoskins Building, Department of Meteorology, University of
Reading, Earley Gate, P.O. Box 243, Reading, RG6 6BB, UK
currently at: Liberty Specialty Markets, 20 Fenchurch Street, London, EC3M 3AW, UK
Louis Williams
Department of Psychology, University of Reading, Harry Pitt Building, Earley Gate, Whiteknights Road, Reading, RG6 6AL, UK
Matthew Lickiss
Department of Typography and Graphic Communication, University of
Reading, Reading, RG6 7BE, UK
Alison Black
Department of Typography and Graphic Communication, University of
Reading, Reading, RG6 7BE, UK
Andrew Charlton-Perez
Brian Hoskins Building, Department of Meteorology, University of
Reading, Earley Gate, P.O. Box 243, Reading, RG6 6BB, UK
Rachel McCloy
Department of Psychology, University of Reading, Harry Pitt Building, Earley Gate, Whiteknights Road, Reading, RG6 6AL, UK
Eugene McSorley
Department of Psychology, University of Reading, Harry Pitt Building, Earley Gate, Whiteknights Road, Reading, RG6 6AL, UK
Related authors
Louis Williams, Kelsey J. Mulder, Andrew Charlton-Perez, Matthew Lickiss, Alison Black, Rachel McCloy, Eugene McSorley, and Joe Young
Geosci. Commun., 6, 111–123, https://doi.org/10.5194/gc-6-111-2023, https://doi.org/10.5194/gc-6-111-2023, 2023
Short summary
Short summary
When constructing graphical environmental forecasts involving uncertainty, it is important to consider the background and expertise of end-users. Using novel eye-tracking methods, we show that where people look and the decisions they make are both strongly influenced by prior expertise and the graphical construction of forecast representations common in presentations of environmental data. We suggest that forecasters should construct their presentations carefully, bearing these factors in mind.
Louis Williams, Kelsey J. Mulder, Andrew Charlton-Perez, Matthew Lickiss, Alison Black, Rachel McCloy, Eugene McSorley, and Joe Young
Geosci. Commun., 6, 111–123, https://doi.org/10.5194/gc-6-111-2023, https://doi.org/10.5194/gc-6-111-2023, 2023
Short summary
Short summary
When constructing graphical environmental forecasts involving uncertainty, it is important to consider the background and expertise of end-users. Using novel eye-tracking methods, we show that where people look and the decisions they make are both strongly influenced by prior expertise and the graphical construction of forecast representations common in presentations of environmental data. We suggest that forecasters should construct their presentations carefully, bearing these factors in mind.
Zachary D. Lawrence, Marta Abalos, Blanca Ayarzagüena, David Barriopedro, Amy H. Butler, Natalia Calvo, Alvaro de la Cámara, Andrew Charlton-Perez, Daniela I. V. Domeisen, Etienne Dunn-Sigouin, Javier García-Serrano, Chaim I. Garfinkel, Neil P. Hindley, Liwei Jia, Martin Jucker, Alexey Y. Karpechko, Hera Kim, Andrea L. Lang, Simon H. Lee, Pu Lin, Marisol Osman, Froila M. Palmeiro, Judith Perlwitz, Inna Polichtchouk, Jadwiga H. Richter, Chen Schwartz, Seok-Woo Son, Irene Erner, Masakazu Taguchi, Nicholas L. Tyrrell, Corwin J. Wright, and Rachel W.-Y. Wu
Weather Clim. Dynam., 3, 977–1001, https://doi.org/10.5194/wcd-3-977-2022, https://doi.org/10.5194/wcd-3-977-2022, 2022
Short summary
Short summary
Forecast models that are used to predict weather often struggle to represent the Earth’s stratosphere. This may impact their ability to predict surface weather weeks in advance, on subseasonal-to-seasonal (S2S) timescales. We use data from many S2S forecast systems to characterize and compare the stratospheric biases present in such forecast models. These models have many similar stratospheric biases, but they tend to be worse in systems with low model tops located within the stratosphere.
Peter Hitchcock, Amy Butler, Andrew Charlton-Perez, Chaim I. Garfinkel, Tim Stockdale, James Anstey, Dann Mitchell, Daniela I. V. Domeisen, Tongwen Wu, Yixiong Lu, Daniele Mastrangelo, Piero Malguzzi, Hai Lin, Ryan Muncaster, Bill Merryfield, Michael Sigmond, Baoqiang Xiang, Liwei Jia, Yu-Kyung Hyun, Jiyoung Oh, Damien Specq, Isla R. Simpson, Jadwiga H. Richter, Cory Barton, Jeff Knight, Eun-Pa Lim, and Harry Hendon
Geosci. Model Dev., 15, 5073–5092, https://doi.org/10.5194/gmd-15-5073-2022, https://doi.org/10.5194/gmd-15-5073-2022, 2022
Short summary
Short summary
This paper describes an experimental protocol focused on sudden stratospheric warmings to be carried out by subseasonal forecast modeling centers. These will allow for inter-model comparisons of these major disruptions to the stratospheric polar vortex and their impacts on the near-surface flow. The protocol will lead to new insights into the contribution of the stratosphere to subseasonal forecast skill and new approaches to the dynamical attribution of extreme events.
Adam A. Scaife, Mark P. Baldwin, Amy H. Butler, Andrew J. Charlton-Perez, Daniela I. V. Domeisen, Chaim I. Garfinkel, Steven C. Hardiman, Peter Haynes, Alexey Yu Karpechko, Eun-Pa Lim, Shunsuke Noguchi, Judith Perlwitz, Lorenzo Polvani, Jadwiga H. Richter, John Scinocca, Michael Sigmond, Theodore G. Shepherd, Seok-Woo Son, and David W. J. Thompson
Atmos. Chem. Phys., 22, 2601–2623, https://doi.org/10.5194/acp-22-2601-2022, https://doi.org/10.5194/acp-22-2601-2022, 2022
Short summary
Short summary
Great progress has been made in computer modelling and simulation of the whole climate system, including the stratosphere. Since the late 20th century we also gained a much clearer understanding of how the stratosphere interacts with the lower atmosphere. The latest generation of numerical prediction systems now explicitly represents the stratosphere and its interaction with surface climate, and here we review its role in long-range predictions and projections from weeks to decades ahead.
Cited articles
Ash, K. D., Schumann III, R. L., and Bowser, G. C.: Tornado warning
trade-offs: Evaluating choices for visually communicating risk, Weather Clim. Soc., 6, 104–118, 2014.
Baguley, T.: Serious stats: A guide to advanced statistics for the
behavioural sciences, Palgrave Macmillan, ISBN: 9780230577176, 2012.
Balcombe, K., Fraser, I., and McSorley, E.: Visual attention and attribute
attendance in multi-attribute choice experiments, J. Appl. Economet., 30, 447–467, 2015.
Bosetti, V., Weber, E., Berger, L., Budescu, D. V., Liu, N., and Tavoni, M.:
COP21 climate negotiators' responses to climate model forecasts, Nat. Clim. Change, 7, 185–190, 2017.
Broad, K., Leiserowitz, A., Weinkle, J., and Steketee, M.:
Misinterpretations of the “cone of uncertainty” in Florida during the 2004
hurricane season, B. Am. Meteorol. Soc., 88, 651–668, 2007.
Correll, M. and Gleicher, M.: Error bars considered harmful: Exploring alternate encodings for mean and error, IEEE T. Vis. Comput. Gr., 20, 2142–2151, 2014.
Fuchs, S., Spachinger, K., Dorner, W., Rochman, J., and Serrhini, K.:
Evaluating cartographic design in flood risk mapping, Environ. Hazards-UK,
8, 52–70, 2009.
Harold, J., Lorenzoni, I., Shipley, T. F., and Coventry, K. R.: Cognitive
and psychological science insights to improve climate change data
visualization, Nat. Clim. Change, 6, 1080–1089, 2016.
Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, R., Jarodzka, H.,
and Van de Weijer, J.: Eye tracking: A comprehensive guide to methods and
measures, OUP Oxford, ISBN: 9780199697083, 2011.
Kelton, A. S., Pennington, R. R., and Tuttle, B. M.: The effects of
information presentation format on judgment and decision making: A review of
the information systems research, Journal of Information Systems, 24,
79–105, 2010.
Kundel, H. L., Nodine, C. F., Conant, E. F., and Weinstein, S. P.: Holistic
component of image perception in mammogram interpretation: gaze-tracking
study, Radiology, 242, 396–402, 2007.
Lorenz, S., Dessai, S., Forster, P. M., and Paavola, J.: Tailoring the
visual communication of climate projections for local adaptation
practitioners in Germany and the UK, Philos. T. Roy. Soc. A, 373, 20140457, https://doi.org/10.1098/rsta.2014.0457, 2015.
McSorley, E. and McCloy, R.: Saccadic eye movements as an index of perceptual decision-making, Exp. Brain Res., 198, 513–520, 2009.
Mulder, K., Williams, L., Lickiss, M., Black, A., Charlton-Perez, A., McCloy, R., and McSorley, E.: Eye Tracking of Vegetable Shipping Line Graph Testing Survey, University of Reading [data set], https://doi.org/10.17864/1947.110, 2018.
Mulder, K. J., Lickiss, M., Black, A., Charlton-Perez, A. J., McCloy, R.,
and Young, J. S.: Designing environmental uncertainty information for
experts and non-experts: Does data presentation affect users' decisions and
interpretations?, Meteorol. Appl., 27, e1821, https://doi.org/10.1002/met.1821, 2020.
Nadav-Greenberg, L. and Joslyn, S. L.: Uncertainty forecasts improve
decision making among nonexperts, Journal of Cognitive Engineering and
Decision Making, 3, 209–227, 2009.
Nadav-Greenberg, L., Joslyn, S. L., and Taing, M. U.: The effect of
uncertainty visualizations on decision making in weather forecasting,
Journal of Cognitive Engineering and Decision Making, 2, 24–47, 2008.
North, J. S., Williams, A. M., Hodges, N., Ward, P., and Ericsson, K. A.:
Perceiving patterns in dynamic action sequences: Investigating the processes
underpinning stimulus recognition and anticipation skill, Appl. Cognitive Psych., 23, 878–894, 2009.
Orquin, J. L. and Loose, S. M.: Attention and choice: A review on eye
movements in decision making, Acta Psychol., 144, 190–206, 2013.
Roulston, M. S. and Kaplan, T. R.: A laboratory-based study of understanding of uncertainty in 5-day site-specific temperature forecasts, Meteorol. Appl., 16, 237–244, 2009.
Rubaltelli, E., Agnoli, S., and Franchin, L.: Sensitivity to affective
information and investors' evaluation of past performance: An eye-tracking
study, J. Behav. Decis. Making, 29, 295–306, 2016.
Savelli, S. and Joslyn, S.: The advantages of predictive interval forecasts
for non-expert users and the impact of visualizations, Appl. Cognitive Psych., 27, 527–541, 2013.
Shepherd, T. G., Boyd, E., Calel, R. A., Chapman, S. C., Dessai, S., Dima-West, I. M., Fowler, H. J., James, R., Maraun, D., Martius, O., Senior, Catherine A., Sobel, A. H., Stainforth, D. A., Tett, S. F. B., Trenberth, K. E., van den Hurk, B. J. J. M., Watkins, N. W., Wilby, R. L., Zenghelis, D. A.: Storylines: an alternative approach to representing uncertainty in physical aspects of climate change, Climatic Change, 151, 555–571, 2018.
Sherman-Morris, K., Antonelli, K. B., and Williams, C. C.: Measuring the
effectiveness of the graphical communication of hurricane storm surge threat, Weather Clim. Soc., 7, 69–82, 2015.
Sillmann, J., Shepherd, T. G., van den Hurk, B., Hazeleger, W., Martius, O.,
Slingo, J., and Zscheischler, J.: Event-based storylines to address climate
risk, Earth's Future, 9, e2020EF001783, https://doi.org/10.1029/2020EF001783, 2021.
Speier, C.: The influence of information presentation formats on complex
task decision-making performance, Int. J. Hum.-Comput. St., 64, 1115–1131, 2006.
Spiegelhalter, D., Pearson, M., and Short, I.: Visualizing uncertainty about
the future, Science, 333, 1393–1400, 2011.
St John, M., Callan, J., Proctor, S., and Holste, S.: Tactical
decision-making under uncertainty: Experiments I and II, Pacific Science & Engineering Group, Inc., San Diego, CA, Technical Report Number 1821, 1–22, https://apps.dtic.mil/sti/citations/ADA378170 (last access: 22 March 2023), 2000.
Stephens, E. M., Edwards, T. L., and Demeritt, D.: Communicating
probabilistic information from climate model ensembles–lessons from
numerical weather prediction, WIREs Clim. Change, 3, 409–426, 2012.
Sutton, J. and Fischer, L. M.: Understanding visual risk communication
messages: An analysis of visual attention allocation and think-aloud responses to tornado graphics, Weather Clim. Soc., 13, 173–188, 2021.
Sutton, J., Fischer, L., James, L. E., and Sheff, S. E.: Earthquake
early warning message testing: Visual attention, behavioral responses, and
message perceptions, Int. J. Disast. Risk Re., 49, 101664, https://doi.org/10.1016/j.ijdrr.2020.101664, 2020.
Tak, S., Toet, A., and van Erp, J.: The perception of visual uncertainty
representation by non-experts, IEEE T. Vis. Comput. Gr., 20, 935–943, 2013.
Tak, S., Toet, A., and Van Erp, J.: Public understanding of visual
representations of uncertainty in temperature forecasts, Journal of Cognitive Engineering and Decision Making, 9, 241–262, 2015.
Tversky, A. and Kahneman, D.: Judgment under uncertainty: Heuristics and
biases, Science, 185, 1124–1131, 1974.
Wickens, C. D., Helton, W. S., Hollands, J. G., and Banbury, S.: Engineering
psychology and human performance, Routledge, ISBN: 9781003177616, 2021.
Williams, L., Mulder, K. J., Charlton-Perez, A., Lickiss, M., Black, A., McCloy, R., McSorley, E., and Young, J.: Understanding representations of uncertainty, an eye-tracking study – Part 2: The effect of expertise, Geosci. Commun., 6, 111–123, https://doi.org/10.5194/gc-6-111-2023, 2023.
Short summary
It is vital that uncertainty in environmental forecasting is graphically presented to enable people to use and interpret it correctly. Using novel eye-tracking methods, we show that where people look and the decisions they make are both strongly influenced by construction of forecast representations common in presentations of environmental data. This suggests that forecasters should construct their presentations carefully so that they help people to extract important information more easily.
It is vital that uncertainty in environmental forecasting is graphically presented to enable...
Altmetrics
Final-revised paper
Preprint