Review of “ Development and Iterative Design of an educational game "Magma Pop” to teach undergraduate fractional crystallization concept” by Saha et al.

Review by F. Jourdan

It’s the first time that I’ve been invited to review an education paper and therefore I’m more used to evaluating research papers, lots of them related to volcanology. That being said, I’m an educator myself teaching 1^st year uni in geology and I truly enjoy the change of pace and topic here. This paper is well written and clear on its goal and its content. I’m therefore recommending publication with minor corrections. That being said, I have one main concern and a few comments after that.

Main concern: I’ve looked throughout the text, and sorry if I missed it, but I did not see any link for the code/game. I don’t know the policy of this journal, but I would assume that if a paper is about a very cool and engaging game, then there should be a mean for the readers to access the game. Otherwise, I the purpose of sharing that with an audience is greatly diminish. A downloadable version should be provided, or a contact if someone want to access the game. If the authors want to sell it, this should be indicated as well, and I would still recommend an evaluation version to be provided.

I have a few comments hereafter, at the discretion of the authors.

L67: a point to consider for an educator and I would suggest rephrasing a little: “reinforcing the learning by repetition” will work for the short term, but unlikely to have people remembering formulas. And why should they? Rather, knowing what elements goes into a given mineral is more useful. For example, Fe, Si and O in fayalite. Useful. The full formula? I don’t even remember it exactly and I use this mineral all the time in my research! The difference might be subtle, but I would argue that the second approach is more likely to stick and not push people away. Who likes learning formula by heart after all? Why I’m saying that, simply because if this phase is too heavily gear toward forcing formulas into students’ heads, that might be a big turn off for them even before the game truly begins. Something to keep in mind. However, in this case, I can see that the formula is provided (and even later in the text, it is provided at every level of the game), so it’s not really to make them learn formula, but rather understand the concept of elements form minerals. Hence my suggestion to rephrase, so it’s clear for the reader.

As a video gamer myself, things that are known to be addictive, are the visual effects and rewards associated with an action. So here, an anorthite crystal is collected a “pop” on the screen showing a small anorthite would be nice. A huge picture of anorthite when the players unlocked the full suite of 5 as well perhaps.  In addition, this make sure that the student understand that the color of a mineral is a give away of its composition. E.g. darker mineral are more mafic and therefore include Fe and/or Mg. Good that characters have been added. What many gamers enjoyed (but not for everyone, so only as an option) is competition. So several well-seasoned teams could compete in completing the task (definitely not the first time going through it).

Level 2: Nice stuff. I would suggest adding a picture of a stratified magma chamber where the olivine end up at the bottom, followed by a mixture of olivine and pyroxene, etc … add the concept of mineral weight sinking at the bottom.

Fig. 4: the answers are in reverse order from the questions listed above (even if the letters can be related). I would suggest changing the order either of the list, or the graph so it goes in the same order.

Fig. 4: I’m really surprised to see that this game did not change much there understanding – it’s clearly a good one (except for question c, where still 75% of the student got it wrong). I assume the correct response is no. so they might be confused by seeing one big piece of gold (effectively 1 atom) and assume it’s valuable. How to fix that misunderstanding though? It would be good to elaborate how that could be improved in the future.

L210: yes, the students should always have access to the formula, or as I was saying, what kind of atoms go into a mineral. Because, is the goal learning something by heart, or understanding a complex process? Obviously, the latter. I saw latter that it was implemented which was a very wise decision indeed.

L87: The pace of the game is described as rapid. In what sense? Is there a timer that force student to do it in a given time (A big no-no I would say), or something else? Could please the authors clarify this point.

I think this game is great. I feel there is some untouched potential though. As for the next level of improvement, I would add a fourth level with two real-life scenarios (or that could be incorporated before in the previous levels). 1. Make a volcano explode. Once the melt is ultra-silicic and saturated with volatiles, it should just explode. I’m not a developer, but I don’t think it’s hard to do. “explode your own volcano”. How cool would that be! 2. You were talking about valuable ore deposits. Too easy, concentrate the liquid and the gold, make it move to cracks at the end, and there you go. Ore deposits associated with volcanoes explained (in a very rudimentary way of course).

What I’m trying to say is that is all theoretical – unless the student will end up working directly with magmatic petrology, that is not going to be a game changer in their career. So it would be good to extend it to real life example (Pompei, etc…)  that would connect the students with a real story and as I’m sure the authors know well, people remember stories even better. So there is more chance that this stick on the long term (again, gamer / teacher talking here).

General comments

This is an excellent attempt to improve the teaching and learning outcomes for a cognitively challenging geoscience topic, all too often presented as a memory task rather than an opportunity for a deeper understanding of the processes involved.

Gamification is a trend in education circles, frequently seen as a cure-all, that often fails to improve learning outcomes because the ultimate structure engages students in the gameplay far more than the desired learning. I am pleased to see the authors recognise this and have set out to create a “serious educational game” and have tried to find an appropriate balance between gameplay engagement and learning outcomes that is to be applauded.

The documentation of the iterative development of this game should be particularly informative and instructional for those who may seek to develop games for other equally challenging topics. However, I think this needs to be set out more explicitly as it is unclear just how different to v1 the v3 product is given the good descriptions of v1's structure.  Perhaps this could be achieved through a more thorough tabular approach that Table 2 does not satisfy.

Unfortunately, as the most senior geoscience courses, the student sample size available to the developers (especially v3) was probably sub-optimal but the quality of the feedback and resultant improvements seems to have made up for that lack of breadth. That said, I think an appendix of all feedback might be valuable and made available, rather than just the quotes selected for the text.

Specific comments

(line 51) The authors state: Major elements usually exist as oxides in the magma and each major element …is represented by different coloured M&Ms. While it is geochemically convenient to express things as weight % oxides (with a large wet chemistry history behind it) that is not the same thing as “occurring as oxides” especially in a melt, even though stoichiometrically it might work. I wonder if that statement cannot be reworded a bit or the oxides bit left out entirely? After all, in Bowen’s reaction series we are talking about the crystallisation of silicates, not oxides.

(line 75-81) Further to that point, we normally teach students that magma viscosity is related to [SiO₄]^4- in the melt (albeit usually given as wt % SiO₂), implying all/most the Si cations and a good proportion of the O anions are already spoken for in the silica tetrahedra anions. I wonder if this does not create some cognitive dissonance amongst students who, on the one hand know the melt is cations and silicate anions and on the other the game starts with the melt just as cations and oxygen anions. While this overly simple model may help students discover and remember mineral formulae (eg: for forsterite is actually a silicate mineral composed of Mg cations and a silicate anion? Further to the general comments above, it is unclear if this 'neophyte' approach survived into v3 or whether the silicate approach was adopted when the levels were labelled basic, intermediate and felsic. Perhaps these changes could be more fleshed out in the text to make it clear just what the final version looks like.

(line 152) Table 1 caption. The multiple choice question is fine, but was it a “one answer only” question or were students told more than one option could be selected? While it may seem obvious (and you explain it in line 164), I think it needs to be clearly stated in the table caption (and maybe even the most appropriate answers identified) for total clarity.

Technical matters

(line 20) Dohaney et al, 2012 does not appear in the references list

(line 21) Jolley et al 2023 does not appear in the references list

(line 21) Dohaney et al, 2023 does not appear in the references list

(line 25) Abdulmotaleb, 2014 does not appear in the references list

(line 35) McGowan et al, 2022 does not appear in the references list unless it is actually  McGowan & Alcott (line 460) in the reference list which has no date

(line 38) Wirth, 2005 does not appear in the references list unless it is actually Wirth 2003 (line 495) in the reference list

(line 42 & line 50) Wirth, 2013 does not appear in the references list unless it is actually Wirth, 2003 (line 495) in the reference list

(line 64) Schonotz and Kurshur, 2007 does not appear in the references list unless it is actually Schnotz & Kürschner, 2007 (line 474) in the reference list

(line 149-151) This sentence maybe technically correct but would be better written another way

Table 1: SiO2 should be SiO₂ and 1000-1050C should be 1000-1050 ^oC unless this is a faithful rendition of the MCQ text given to the students

There are many papers in the reference list that are not cited. I have only listed papers that are cited but not listed or apparently mis-listed.

I do not have the ability to ensure every paper cited or listed is appropriate but I encourage the authors to ensure the papers they do directly cite are correctly referenced re spellings, dates and titles. I also wonder if the many uncited papers listed are required for the paper even if they are relevant. If they have specific relevance should they not be cited?