In developing the course, we were engaging in design-based research see, for example, . This branch of education studies simultaneously develops, tests and improves an educational module, proceeding over iterative cycles. Chp. 16.10 link it to engineering studies, where prototypes are developed, tested, and the feedback is applied to a new development round of the product. While our analysis for this study rests on the 2024 course edition, we have taught the course four times in total (2022–2025). During the first two years we incorporated participants' feedback (see Sect. ). Then we took the third year (in 2024) as an opportunity for a more thorough evaluation of the course concepts. The design-based research framework fits our approach because it formalises our two-fold goal of designing and researching the teaching and thereby fruitfully combines our two interlinked roles of teachers and designers. In design-based research, the “agenda of the designers is seen as a positive force rather than a threat to validity” . Moreover, the approach is interventionist, for example allowing “tweaking the intervention to better match the design intent mid-implementation” rather than sticking with an ill-suited design in order to keep study conditions constant. We made use of this when improving the course in between editions.

For the third cycle, we set out to not only evaluate and improve the course in terms of direct feedback, but wanted to dive deeper into understanding the students' learning progress. Our goal for the course has been to teach both numerical (climate) modelling and its critical reflection. The combination of natural sciences with philosophy of science and STS has been a thought-provoking and challenging experience for us. We were motivated to give students in the course a similar intellectual experience as well as a holistic picture of climate change modelling as a socio-scientific issue from the start . During the first two cycles, we have noted incidents that hinted at individual students undergoing a profound change of their perspectives and conceptions. Therefore, in the third cycle we wanted to study what that process looks like and whether more than single students were undergoing it. Additionally, we were interested in which modules in particular facilitated the learning process and where students faced challenges integrating modules with their thoughts. Accordingly, we formulated our research question as:

In our interdisciplinary course on climate modelling, what does the students' learning look like and does it align with envisioned learning outcomes? What thought processes does the course trigger? Do we see evidence for change in students' thinking about climate models?

In choosing the assessment method we were guided by the following considerations.

As time for the course is short, the course should not be interrupted by extra activities, but these should be integrated into the course flow.

We thus opted for the iterative direct assessment of students' perception changes in a reflective exercise. Asking students for changes in their thoughts and opinions directly demands a high level of self-reflectivity on the spot. To ease them into this task, we chose to ask first what thoughts or new ideas were going through their head after a specific module (posteriori). In a second step, they were asked to add what they had been thinking about that topic beforehand (a priori). The direct confrontation was thought to be helpful for the students' reflection, but bears the threat of increasing the “good-subject effect.” For each assessment we asked:

a posteriori: “In relation to the material covered in the last class session, note which ideas, insights or concepts are spinning in your head now. You can do that in the form of, for example text, notes or pictures.”

Our position and experiences shape the knowledge that we produce as researchers see, for example,. Thus it is important to make them explicitly transparent p. 225. In this case, our role in this research is shaped by our deep engagement and identification with the NAka project, as we have both been involved multiple years, and one of us is a former participant as well as former project leader. During each course year, the students and we build a relationship that is conducive for teaching, but also colors our approach to the students as research subjects. It may also enhance the participant bias . In addition, we have teaching training and experience, but are by no means educational (research) experts. Thus, our goal with this study is not to provide an objective evaluation, but rather to showcase a course approach that has proven itself in practice. Rather than hampering the results, we think our enthusiasm and the relationships we built play a large part in the success of the course.

Figure  and Table  give an overview of the course schedule. It is divided broadly into two themes: the first theme is concerned with the mathematical and physical aspects of numerical modelling and its application to climate models, while the second theme reflects model building critically and discusses the development of climate models in a socio-historic context.

The first topical module introduces numerical modelling as a general method and its various applications. To motivate the relevance of numerical modelling we showcase examples from various scientific disciplines in the form of pictures, brainstorming possible subjects and their diverse goals and challenges with the students. Next, we define the modelling of dynamical systems more formally and agree on a common nomenclature. This is achieved via a student presentation showcasing the differential equation (DE) of a simple physical system. To practice the newly learned terminology for the different types of DEs and their constituents we use further examples of DEs describing dynamical systems in natural sciences.

The next module is concerned with the analytical solution of the DEs. Given the varying mathematical education of the students this is a challenging topic. Therefore, we solely rely on finding a solution by means of “good guessing”. The difficulties encountered during the exercise and the fact that the analytical method is only limited to a small collection of simple systems motivate the use of numerical methods for the remainder of the course.

As the most basic discretisation method for solving ordinary DEs numerically we introduce the Euler method (forward and backward). For practice we let the students solve the logarithmic spiral only using pen, paper and a calculator. Since this example was already part of the analytical exercise, the students were able to compare both methods. The important lessons are: (i) the numerical method is not exact as it deviates from the analytical solution and (ii) the numerical method takes much effort since many more computational steps are involved. This is why we ultimately resort to computers for automating the calculations.

Our course has no requirements on prior knowledge of programming. Therefore, we teach the basics from the ground up using a simple tutorial notebook that includes exercises. As we do not have the time for an extensive programming class we follow a learning-by-doing approach in the rest of the course and rely on more experienced students to help less experienced ones. Our choice of programming language is Python as it is easy to learn and widely used in the scientific community.

Then, we form the basis for understanding the climate system and its numerical modelling in climate models. This is achieved via first collecting students' prior knowledge of the Earth system and their interactions in a black board diagram. Additionally, there is input on climate change, climate model structure and the uncertainties in climate modelling via student presentations to dive deeper. Climate models are explored hands-on by showing the students actual climate model code and via the IPCC Interactive Atlas , which illustrates real model output for different variables and scenarios. Furthermore, the students write their own program to model a simplified version of the greenhouse effect, which illustrates the usefulness of numerical simulations to study processes in the Earth system.

To further practice numerical programming and learn about model building the students work on programming projects in groups. We offer a diverse set of topics (see Table ) related to the Earth system that highlight different aspects of dynamical systems, e.g., feedbacks and chaos, and also different technical intricacies of numerical modelling, e.g., the comparison of alternative discretisation schemes for partial DEs. Half way into the programming projects we ask the students to reflect on their efforts: Why do we model? This question bridges to the second theme of the course about critical reflection of model building.

This second thematic part covers the goals underlying climate model development, issues in the interpretation of climate model results and their development as embedded in the societal context. For example, by constructing a timeline from given index cards, the students dissect the co-evolution of climate models alongside relevant historical events for the material, see. The subsequent first silent and then guided discussion reflects how the historic context has influenced the field of climate science and thus how models and for example views of a global Earth were co-produced . In another exercise, the students get to know three main motivations or visions for climate model development by assigning quotes or methods to the vision categories. These have been developed by and summarized by as the representative, predictive and heuristic vision. These visions put the focus on the model being a copy of the real system, providing accurate forecasts, or on being used as a tool to generate understanding, respectively. While these visions can work together, they may also lead to conflict, for example where more detailed models that are more representative become too complex to understand, thus decreasing their heuristic utility . and have explained some problems of climate modelling from a philosophical perspective, such as distributed epistemic agency and generative entrenchment. These texts serve as the basis for a group work where students read the texts in groups and then present them to the others in a creative format. An example of a particularly vivid display is shown in Fig.  and described further in Sect. . After discussing long- or outstanding issues in climate model development, we find it important to circle back to the question of why one can trust many climate model results after all. has written accessible elaborations of the reasons that serve as the basis for one student pair's presentation. The course content ends with a “fish bowl discussion” of climate scientists' position in the climate change debates. In a “fish bowl discussion”, students are divided in groups and get some input for a particular position they are asked to represent. One student of each group then sits on the podium and represents this position in the discussion, but at any time another member of the group can leave the audience, tap on the discussant's shoulder and take up their position on the podium, allowing everyone to participate and bring fresh arguments to the table. In our case, the positions ranged from disinterestedness in public discussion to activist positions. While students can use knowledge gained in the course to back up their arguments in the ensuing discussion, the topic circles back to the idea that climate models are a product of and feed back into our society.

Each of the four years that we taught the course (2022–2025) offered an opportunity for improvement, based on our own experiences and students' feedback. After initial struggles with the analytical solving of differential equations, the basics of numerical modelling and model building seemed to always be well understood by the students. Integrating the philosophical perspectives and STS content was more challenging. In the first year, we separated the numerical modelling from the “Interaction with society and critical reflection” as a first and second thematic block, but in the following years we integrated the two approaches more. The integration serves to have the understanding of both perspectives benefit each other, with parameterisations being a key component of model formulation and reflected in the representative modelling vision, but also a basic reason for modelling uncertainties. Also, the integration allows to mix methodologies, with more discussions and text-based work in the “Interaction with society and critical reflection” part of the course. For the same purpose, we have increasingly dispersed the students' presentations throughout the course. Appendix  provides the course schedule from the third iteration (2024).

Figure  shows the topics that participants included in their reflective exercises. After inductive coding, we found that main topics correspond to topical blocks in the course program, for example model problems, visions, or the concept of a culture of prediction , and thus assigned those deductively (compare Table ). Regarding our research question, this already shows that the learning aligns with the teaching goals. The frequencies of topic mentions also relate to when the reflective exercises were conducted (see Fig. b): for example, the topic group “1. Dynamical systems” was not explicitly sampled, because the emphasis of our evaluation was on topic block 3 and 4. Regarding climate model development – one of the most prominent topics mentioned – students highlighted topics that were explicitly treated in the course, such as the model structure or parameterisations . A particular piece of research that has left an impression on some of the students is climate model geneaology , meaning how different (generations of) climate models are interconnected, for example by one model being built on the basis of another. In particular, have provided a display of models' relationships in their Fig. 2, which we have used to visualise both the multitude of models, the countries of those who develop them and the interrelationships. This visualisation speaks to the students as they have repeatedly referred to it during discussions in the course.

One prominent topic in the participants' responses are the different modelling visions. The understanding that there are multiple visions that may lead to conflict emerges directly from the corresponding exercise conducted in the course (see Table ). However, two students also expressed the idea that the different visions lead to more diverse science, i.e. multiple approaches being followed. While this positive understanding was not an explicit part of the exercise, it corresponds to arguments in favour of climate model hierarchies as brought forward in the literature .

Another topic is that of how science works. That this topic came up is surprising to us because it was not explicitly treated in the course. Here participants viewed science as a practical job, the scientists as people, and scientific process in general as not being objective as discussed by. For example, one participant commented on the “chaotic scientific work” and elaborated that “the everyday life of science is by far not as polished as papers can make it seem [TN19, RE2].” In particular, one participant seems to have imagined themselves in the climate modelling job, asking “How many feelings of success does one have in climate modelling? [TN03, RE1]”

There are no clear misconception in the responses. However, the pronounced presence of the “scenarios and manipulation” code topic strikes a cautious note. This topic arose out of the course work with the IPCC Interactive Atlas . Students were asked to think of a question to investigate with the simulation results and plotting capabilities available on the platform. In the discussion of their results, we paid particular attention to how the different time frames and climate change scenarios they used can influence the answers to their questions. Our treatment of the influence of scenario choices seems to have combined with pre-conceived ideas of manipulation to form the idea that scenarios can be or even are used to manipulate: “by choosing different scenarios one can easily manipulate humans” (N13, RE2). Informal conversations with students at the NAka 2025 pointed us to what these pre-conceived ideas could be about: students were sensitive to fake news and manipulative statements, seemingly because of frequent treatment of these issues in school see e.g., also due to the growth of right-wing populism in Germany . While scenarios do have a large influence that should be questioned, manipulation is not what climate science uses them for. Here we recognize an issue that STS have had to grapple with: on the one hand, from a constructivist point of view one comes to criticise the power of science and its human foundations (see, for example, , , and , for an explanation of constructivism). On the other hand, most critics do recognise science's results as true and do not wish to imply that for example climate change is not real. This is a delicate balance to be struck see, for example,. From the students' responses we saw that they conflated subjectivity within the scientific process with more or less deliberate manipulation. Consequently, we took more time in the next year to introduce scenarios more rigorously, detailing their scientific basis as well as the choices embedded in their creation and the need for careful interpretation of the scenario used in a study.

The second part of the exercise also targeted changes in students' perceptions, as they had to describe how they thought about the mentioned topics before the course or before the last course modules. Fig.  displays the results of that exercise. The answers to the different rounds of the reflective exercise are combined here, because answers usually do not refer to (topics of) past modules covered in other rounds. In general, students reported less knowledge before the course and simply not having thought about some issues before (for example “I didn't know so clearly that there are also many negative feedback loops (I had only heard of the halting of the Gulf Stream before)” (TN16, RE2)) , which supports the course's goal to introduce knowledge from beyond what is present in high-school curricula. When they knew a concept from before, some students said that giving it a name makes the concept more concrete. Establishing a language to grasp concepts and talk about them is one part of what social science knowledge can do for natural scientists, as for example explain. With regards to climate models, multiple students reported to have found them “unimaginable” (P3, RE1) before, but that their idea of them became more concrete. In that sense, the course allowed them to un-box climate models and build an understanding as a basis for interpretation, as students demonstrated with the exercise displayed in Fig. . The understanding that climate models are complex goes hand in hand with this unboxing. One students said they had been aware of complexity before, another said they had underestimated it, and more had been unaware before. Human influence was a topic that was more frequent in “a posteriori” comments, where students realised that scientific endeavours such as building or running a climate model are subject to values and human influences, again in line with the course content.

A particular insight into participants' thoughts during the course came from the questions they asked themselves during the reflective exercise. During the first exercise, one participant asked (arrow present in their notes):

How much of climate modelling is logical and easily explainable and deducible? → How much can/could we implement in our own climate model? (TN04, RE1)

Others took this question further, wondering whether such a model would even be useful: “What are the advantages of more precise climate models? Does that influence human action? → Aren't current statements enough? (TN10, RE3)” The course-block on the historical evolution of climate modelling brought up the question of how it will continue: “History of (climate) science extremely interesting → for me it raises the question: where are we going? How will these sciences evolve? (TN03, RE3).” These comments and questions highlight that at least for some students the course prompted a thought process on the goals and future of climate modelling.

A particularly vivid display of students' thinking about climate models emerged from the exercise on climate model problems. Three groups were asked to read different excerpts from and , discuss them and present the content to the other groups, in a creative format. The group tasked with the excerpts from decided to turn a small spare room into a climate model display. Figure  shows how they used flipcharts as model components, ropes to link them, and annotated the ropes with the linking elements. For example, aerosols would be linking the atmosphere and radiation component of the model. Model components were written in different languages to represent international development distributed in time and space. They purposefully included a mistake or bug in the model , and the overall entanglement of the ropes served to represent the complexity of the model.

The reflective exercise we conducted revealed topics students were debating at the time of the exercise. On the one hand, these mostly aligned closely with the course content, so it was difficult to identify students' personal thought process amidst the general course progression, and thus our contribution to that research question is limited. On the other hand, many codes of the students' answers only had one count, as students mentioned a wide potpourri of statements among each other. That each student takes away something different is of course an interesting feedback and learning for the teachers. It also confirms that the method was suited to allow for a wide range of responses and gave students the freedom to detail their own thoughts (which then still were closely aligned with the course progression). One weakness of the method was that students often did not spell out the “a posteriori” when naming the “a priori” and vice versa. For example, they said climate models were unimaginable before and that only implied that it is not unimaginable afterwards anymore. A clear caveat to our approach are the various forms of participant bias. For example, in an assessment setting, participants are prone to answer according to how they think their viewpoint and ideas should have changed from our point of view. We addressed this by explaining the purpose of our study to the students and encouraged their own scientific curiosity in the reflection. In addition, the plenum discussion following the exercise provided us the opportunity to analyse their expressions in more context. The exercise did encourage students to reflect, as the nature of responses changed from content and feedback related notes to reflections with more exercise rounds. Reflective group discussions or a reflective essay may have provided more in-depth views of students' reflections.