Review of ‘A snapshot sample on how COVID-19 impacted and holds up a mirror to European water education’ by Fischer and Tatomir

Review by Rolf Hut

The authors report on a survey they conducted among teachers of higher education courses in hydrology to gauge the impact that the COVID-19 pandemic and associated (lock down) measures had on teaching and learning relating to hydrology in higher education.

This is an important topic: education to new generations (of hydrologists) is an important part of geoscientific communication and sharing the insights from fellow educators might help the hydrological community to either be better prepared for future similar situations (I really really hope this will not happen…) or to learn lessons from this situation for education in general.

I do think that there are issues with the setup and the reporting of the study that should be addressed before the current manuscript is fit for publication. Below I will separate those into major comments, that deal with the overall setup and reporting, and minor comments that deal with particular phrases or lines in the document.

Major comments

On sampling and conclusions

The authors rightfully take care to call their survey a ‘snapshot’. Only 28 people responded to the survey. The authors claim that this is a ‘14% response rate’, which means they consider the 200 people to whom they sent the initial survey to be their frame (or ‘target group’). Given the membership number of, for instance, the EGU hydrology section, the 200 people contacted seem like a sample of the population that is the intended target of this research: people teaching hydrology at higher education institutes. Please provide a definition of who the ideal ‘target group’ for the survey are: all people involved in teaching hydrology at universities? Given that target group, in the results section please reflect on how representative this sample is for that target group (are all career levels represented? Genders and regions? Etc.). Based on the representativeness of the sample, please make a statement on how much this snapshot can be extrapolated to the wider target community.

The response within those 28 does not seem representative of the profession of people teaching hydrology, given for example the skewed distribution of countries. Yet, despite this, the authors make broad claims on the impact of COVID-19 on teaching hydrology, for example in the abstract: “Hence the important knowledge of process understanding in hydrology will be missing for generations of hydrologists”. These broad claims do not follow logically nor statistically from the results of the survey.

I recommend that the authors make a thorough assessment of all the claims they make in the manuscript and check if it is supported by the data they present (including a proper statistical analyses in that case) or if it is supported by pre-existing literature, in which case this should be cited.

On hydrology in the broader context of higher education during the COVID-19 pandemic

The literature cited contains a long list of papers related to the teaching of hydrology, mainly from the Seibert et al 2013 special issue on the topic. This nicely frames the research in the literature on education in hydrology. However, nearly all literature cited is written by people with a (research) background in hydrology, who also teach. There is a wealth of knowledge from researchers who study (the effectiveness of) educational approaches. Especially since the pandemic, numerous papers have been published detailing the impact of COVID-19 on education of all levels. Given the low number of responses (see above) I think it is even more important to relate the results to what is already known of the impact of COVID-19 on education in general to see if the results for this survey among hydrologists align with the broader view seen across all educational topics and level, or that the results show a remarkably different image for hydrology. I recommend that the authors review this part of the scientific literature and add a reflection on the impact of COVID-19 on education overall in the introduction. In the results and discussion the authors can than reflect on the similarities between their survey results and the impact of COVID-19 on other fields of education.

Related to this: I believe the study by Wagener 2007 can be used more explicitly in this work as a starting point for hypothesis testing. Basically the overview of Wagener is taken as H0, the situation before COVID-19 and this survey is used to test if the situation has (significantly) changed. The significant part might be an issue with only 28 respondents though...

Minor comments

On my own publication

While I appreciate the citation to Hut 2020, I would like to stress that this publication deals with transferring a hands on physics class from an on-campus, indoor in the lab, situation to an at-home situation. This being a physics and not a hydrology course makes the mention of it feel somewhat strained. I would definitely not call it ‘field work’ as is done in table 2.

Other minor comments

• L53-54 Venhuizen 2019 (on which I am co-author) does not make any statements that education on hydrology is ‘flooded by jargon’, merely that the jargon used by hydrologists might be mis-understood by non-hydrologists.
• L132 in the spirit of Open Science, please share the scripts that generate the results
• L134 please justify the use of a word cloud as a scientific visualization tool, or use a more appropriate algorithm to extract meaning from text based answers to the survey
• L170 ‘less known’. This can not be concluded: teacher may very well know of other methods but not employ them for a variety of reasons.
• L171-173 I have problem with the word “conservative” here as it communicates (to me) a value statement that certain, older, types of education are less effective than others. I always argue that there is a type and a place for each type of education, including traditional lectures. I would ask the authors to reflect the distribution of lesson activities and compare it with how things are done in other fields of higher education teaching, if they want to make a statement like this.
• L176 I think ‘lectures’ should be removed: all teaching, including lectures, went online
• L181 I don’t understand ‘critical’ in this sentence
• L181-197 combine both survey results (figure 5), literature information (wagener 2007) and discussion on methods (timing of the survey). For this paragraph, but more broadly for the entire manuscript, it would help to clearly separate these:
  • what is known about teaching hydrology from literature before your survey?
  • What does your survey add to this knowledge?
  • What are the limitations / caveats of your survey?
• L198-202 a great deal of literature is available on online learning before COVID-19, mainly from the Open Universities of this world (work of Peter Sloep, among others). Please link results to this work where appropriate.
• L198-208 please mention how often these things were mentioned in the survey
• L209-210 “Despite … to COVID-19” This is speculative and either needs a reference to back it up or be moved to a separate discussion.
• L211 “difficult to tell whether students reached their learning goals” this does not follow from figure 6C, I guess figure 7e is meant? If so than this is a good example of the earlier point on number of responses and significance. If we look at the situation before COVID-19, how many students did reach their learning goal? For how many students was this “difficult to determine” (I guess a rather large percentage given how hard it is to design assessments that actually measure learning). If we draw randomly 28 teachers from the pre-COVID-19 situation, how likely would it be that 39% indicated that students learning goals were hard to assess?
• L218 needs a reference to back this claim up
• L220-221 needs a reference to back this claim up
• L228-231 maybe mention something about the region the respondents were in here, since that greatly influenced which (if any) government restrictions were in place at what time
• L234 ‘cover’ should be replaced by ‘come from a selection of hydrologists from’
• L235 ‘needs to be seen’ has multiple interpretations possible. I would suggest replacing by ‘is uncertain’
• L235-237 I would remove this since I do not think this conclusion can be drawn from this snapshot.
• L240-272 this whole section is overconfident in the results of the survey and should be re-written with this in mind. For example: “Due to COVID-19 the important knowledge of process understanding in hydrology will be missing for at least several cohorts of hydrologists” can not be concluded from this limited survey and can only be tested a few years after this pandemic has settled and broad research on this is conducted.

Despite that I think that the topic is of interest to the readership of GC, I have pointed out in my opinion some serious flaws in the research and its reporting. I think that with careful rewriting the readership of GC in general and those teaching hydrology in particular can learn something from the results of this survey, so I would recommend major revisions to this document before publishing in GC.

Best regards,

Dr. ir. Rolf Hut

Review by Dr. Nicholas J. Kinar

University of Saskatchewan

This is a landmark paper that clearly indicates how the COVID-19 pandemic affected educational hydrology-related initiatives in a European context.  The historical importance of this paper cannot be understated, since the authors carefully and methodologically indicate why changes occurred and how the pandemic both identified and created rifts and challenges for both instructors and students.  The paper indicates that fieldwork and other in-person opportunities were influenced and provides the start of a foundation for other papers related to geoscience communication.  This is a good paper that I anticipate citing in the context of my own work, and I recommend only some minor revisions before publication.  I thank the authors for documenting and identifying challenges and opportunities; this is an excellent paper that is a good contribution to the research on the scholarship of teaching and learning over the past few years.

The authors should consider defining in the abstract what is meant by “holds up a mirror” to educational opportunities. 

Line 26: A “spirit of optimism” is not clear.  Please re-write this sentence to indicate what is communicated by this term.

lines 35-36: What are the types of challenges as per Wagener et al. (2012)?

line 54: What is meant by “flooded by jargon”?

line 91: “make the hydrology alarm bells ring” is unclear and please re-write this sentence.

Section 2 Methods: What was the survey methodology used for survey design?  Gideon (2012) might be useful to cite here.

Gideon, L. (Ed.): Handbook of survey methodology for the social sciences, Springer, New York, 520 pp., 2012.

line 124: What is “googleForm”?  Is this the Google Forms product from Google?

line 146: What is meant by “response rate” in this context?  Is this a percentage as per the total number of universities or people who received the survey?

line 164: What is meant by “state-of-the art”?

line 212: What is meant by “CATs”?  Do these involve “clicker” hardware and software?

line 266: Please provide more context on Bloom’s taxonomy.  Not all readers are immediately familiar with this taxonomy.

line 268: What is meant by “damage” in this context?

The authors should provide all of the survey response data as Supplementary information so that the readers of the paper can better understand how the survey was conducted.

Figure 4: What is meant by “exotic” teaching methods?  The difference between methods (traditional, practical and exotic) should be clearly outlined in the text.

All figures involving word clouds: Please discuss in the text how the word cloud contributes to the presentation of the research in the paper.  What is the meaning of larger words relative to smaller words.