We thank the reviewer for their time and helpful comments regarding our manuscript. We believe that with the described additions below to better explain our study, particularly for those that are unfamiliar with TikTok or similar social media analyses, our manuscript will be suitable for publication and of significant value to the science communication community.

Given the total opacity of the TikTok algorithm, our study seeks to elucidate patterns and trends related to reach and engagement of geoscience content on TikTok so that science communicators can find the most success there. More so than any other social media platform, content discovery on TikTok is algorithmically-driven, and our study uses all analytical data metrics available on TikTok to provide insight relevant for science communicators. We specifically analyze (1) reach [video views], (2) video view duration, and (3) video engagement in assessing the impact of each video. We seek to maximize all three factors: (1) videos seen by more people increase the audience impacted by the content, (2) the longer people view a video indicates greater interest in the video and subject, and (3) engaging with the video (like, comment, share) demonstrates additional interest in the video and subject, all reflective of successful science communication. 

The videos we created for the Terra Explore TikTok were part of a pilot project to test producing geoscience video content on TikTok. As such, we aimed to produce videos of various topics/styles, durations, with different hashtags used, and different posting times/dates in order to sample parameter space. Although there was not a systematic design to every video, these videos still provide important insight and much needed data for science communication efforts on TikTok. As it is difficult to isolate singular variables, we use the similarity of content and format in GMV (ground motion visualization) videos to more clearly evaluate individual factors. We acknowledge that algorithms may change over time as platforms evolve, but we are not aware of evidence that TikTok is changing its algorithm so rapidly as to invalidate attempts at analysis.

1. We survey all available data metrics from TikTok to discern any notable trends in video reach and engagement, as is common with research examining content on social media platforms (see Habibi and Salim, 2021 and Wang et al., 2022 (Geoscience Communication)). We will more specifically describe the purpose of each parameter that we are looking at and what it indicates, as well as our associated hypothesis with how it relates to algorithmic discovery. As an example, social media algorithms are generally believed to reward videos that have a high viewer retention (how long someone watches the video) and high engagement rate by showing the video to more people (as these videos keep users engaged and on the app/platform for longer) (Klug et al., 2021). However, short videos on TikTok (~15s) can very easily yield high viewer retention rates, which doesn’t necessarily translate to high views or engagement. Understanding the data related to each metric is thus important in deciphering what characteristics are likely to make a successful video. Beyond understanding the algorithm itself, a high average view duration and a high engagement rate indicates that the audience is interested and invested in the video content and are extremely useful metrics for evaluating how engaging and successful the video content is.

We will remove the figure depicting gender of followers over time, as a text explanation will suffice.

1. We strongly agree that statistical analysis is needed in this study and will greatly strengthen our analyses. We are working on performing correlation analyses for all the relationships we evaluate (including r, r², and p values to determine statistical significance).
2. TikTok provides information about the top territories of an account’s followers and individual video views by region. ~85% of the Terra Explore account’s followers and views are from the United States, which is why we defaulted to a more US-centric view. However, we will include this information related to viewer/follower region and will analyze any videos that have a significant proportion of views from outside the US. We hypothesize that TikTok heavily mines data based on a user’s location and often shows video content local to that viewer. Post time optimization is typically linked to the location where the majority of your audience is, and hence would apply to ‘local morning’ vs ‘local evening.’
3. While our videos posted were part of a pilot, we believe that they provide useful information for science communicators, especially given how limited any existing studies of science communication on TikTok are and how untapped the potential of TikTok is.

We will adjust the title of the manuscript so that it more accurately reflects the content. 

Line 16: “Viewer retention” (also ‘audience retention’) is a standard metric in social media for the average percentage of a video people watch. However, to keep this term consistent with our figures, we will refer to it as the ‘average video view duration (%)’ throughout. 

Line 20: The “For You” page is the Proper Noun name for this feed on TikTok. In the abstract, we define it as “TikTok’s algorithmic recommendation feed.”

Line 22: We will qualify ‘viral’ as high views and engagement.

Lines 53-55: As described in point 1, we will more thoroughly outline our methodology and describe the importance of each metric we analyze. We analyze engagement, as videos with high engagement are hypothesized to be rewarded by the algorithm and will continue to be shown to more people. This cycle of increased engagement and reach thus increases the impact potential of the science content. High engagement rates also indicate an increased interest from the viewer in the video, demonstrating the impact and ‘success’ of the video. 

Lines 58-77: We can include additional context regarding the global distribution of TikTok users.

Line 67-68: TikTok provides no information regarding their AI algorithm or how it works, hence why studies like this are important. The information we provide in this section is essentially the extent of all information TikTok publicly makes available. Users can indicate they are ‘not interested’ in a video, but there is no way to choose topics of interest other than watching videos on the app (or skipping them) and liking videos that they enjoy.

Line 77: We recommend downloading TikTok for a brief user experience. Sounds are background audio clips that can be re-used by multiple users and are separate from hashtags. 

Lines 85-93:  The lifetime of a TikTok hashtag is essentially infinite. You can click on a hashtag and see all videos that have ever been created that use that hashtag. Any person can use any hashtag at any time. As we discuss later in the paper, hashtags on TikTok have a fundamentally different ‘function’ than they do on platforms like Twitter. People generally do not ‘search’ for hashtags on TikTok like they do on Twitter. We hypothesize that hashtags are primarily used in TikTok’s AI to provide additional information and context for the video when deciding who to show it to with the algorithm.

Lines 104-105: We do not say that it is impossible to have experiment videos related to geosciences, merely that they are generally less common and/or easy to produce compared to videos of chemistry and physics experiments. A survey of educational science content on TikTok demonstrates an abundance of videos of chemistry and physics experiments and near total absence of geology-related experiments (which we find is more so related to the ease with which at home or classroom chemistry/physics experiments can be filmed and shared on the app).

Lines 123-124: The video presenters are the existing communications staff at our respective organizations. The male presenter was only able to create two videos during this duration, hence we are unable to analyze video metrics based on the gender of presenter, although this is work we would like to do in the future.

Section 4 (general): As it is very difficult to vary just a singular variable when creating TikTok videos, we use the GMV (ground motion visualization) videos as a way to most clearly isolate variable trends with posting. All the Terra Explore videos were created during a pilot project, in which the goal was to test and produce different types of geoscience content on TikTok (different topics, durations, hashtags used, time and day posted, etc.). Although there was not a systematic design behind every single video posted, that does not make the analyses of these videos any less meaningful. 

Line 175: We will include UTC with our time zones.

Fig.2: We will delete this figure.

Fig 3: ~85% of viewers of Terra Explore are within the United States, hence the majority of views come from ‘local’ time zones. The concept of post time upload is related to optimizing the upload of your content for when the majority of users/followers will be active on the app so that it is seen by the highest number of people. All our most highly viewed videos (>90,000 views) were posted between 7 am - 2 pm MST (UTC-7), which allows more time for the content to be live and assessed by the algorithm before the number of active users drops off overnight. However, the lack of a distinct trend in this plot indicates that post time is but one of many factors that may impact the success of a video. This concept is also not to indicate that the morning MST is the most ideal time to upload, rather that it is the local time of the majority of the audience.

Fig. 6: We will delete this figure.

Lines 215: “Viewer retention” is the common terminology in social media. However, to keep this term consistent with our figures, we will refer to it as the ‘average video view duration (%)’ throughout. 

Fig. 7: We will add a more thorough explanation in the text for those that are unfamiliar with metrics used in social media analyses. Traditionally, yes, you would expect to see a relationship between video views and average view duration, as videos with a high avg. view duration are hypothesized to be rewarded within the algorithm and shown to more viewers, thus yielding higher views (particularly on platforms like YouTube). A high average view duration also indicates that the audience is more interested in the video content and that it holds their attention rather than scrolling to the next video, thus providing a useful gauge in how effective the science communication video is. However, because TikTok can support such short videos (~15 s), those videos can very easily yield a high avg. view duration, but that doesn’t necessarily mean they will also have a high number of views. We see videos with the highest number of views (>90,000) have avg. view durations of >40%, indicating that this is the value creators should aim for.

Fig. 8: The higher the engagement rate, the more people are interacting with a piece of content, thus showing higher interest on the side of the viewer (see Habibi and Salim, 2021, another study that measures user engagement of educational science content on TikTok). As is hypothesized with how algorithms work, content that is interacted with more will be shown to more people, thus yielding higher views. What is interesting on this plot is that even videos that did not yield a high number of views still yielded high engagement rates. Here we see no observable trend between number of views and the engagement rate on the content. It could be that a video needs both a high engagement rate and a high avg. view duration to be shown more in the algorithm and receive more views (see Figure 9).

Fig. 9: We find that there is no clear relationship between the average view duration and engagement (two factors that are likely to increase video reach in the algorithm). Thus, just because a person views more of a video does not necessarily mean they are more likely to interact with the content. However, the average view duration here is also linked to the length of the video, as ~15 s videos can much more easily yield a high avg. view duration than can a ~150 s video (Fig. 10–the longest videos actually have consistently high engagement rates, especially for shares). 

Section 6.4: Explained in reply to Fig. 7.

Section 7: There is essentially zero transparency to TikTok’s algorithm, hence the importance of this study and others like it. The existing literature related to analyzing content on TikTok and the algorithm is very limited (see Klug et al., 2021, which we will discuss and add a citation for). This study is thus important because it provides additional data and insight for the very little that has been analyzed of science communication on TikTok. We will address in the text that algorithms can change over time, and thus it will be useful to conduct future surveys over time. However, algorithmic changes that fundamentally impact how the app functions are unlikely to occur over short periods of time. 95% of teens in the United States say that they have access to a smartphone (Pew Research Center). Thus there is almost no exclusion to the technology, demonstrating that TikTok has nearly unlimited reach potential. We have addressed audience demographics in previous comments.

Lines 405-412:

1. We have addressed these concerns all in previous comments (see comments to Figure 3). 
2. We do not intend to suggest that every topic be related to a newsworthy event or place-based geology, rather that these are ways in which to maximize the algorithmic impact of the video content, thus reaching the highest amount of viewers and sharing the science with a broad audience. These are also tools to make the topic communicated relevant and relatable to the audience. As we demonstrated, even videos that had lower views yielded high engagement rates, which indicates that the content was still impactful, even if it was received by a smaller audience. We find that demonstration videos < 30 s are an impactful way to communicate topics, as are lecture-style videos ~40 - 120 s in duration. We include this discussion of gender and this recommendation as we hypothesize that there may be inherent sexism baked into the TikTok algorithm, where accounts that are identified as female are shown less or are not shown science-related content on their “For You” page. “The recommendation would be merited if the strategy is to include more gender-related topics to gain more female followers, who then view videos of other topics is precisely what we suggest and recommend. A nuance to note is that videos from an account a person is following are also often shown on the ‘For You’ page. Hence, even if the video is viewed by an account follower, the view will still be counted as coming from the ‘For You’ page.
3. We re-emphasize here that views on TikTok generally do not come directly from hashtags (like one might look up a hashtag on Twitter), rather that hashtags are tools to provide additional algorithmic context for the video (especially beneficial if highlighting a location, e.g., #california).

We thank this second anonymous reviewer for their time in reviewing our manuscript and for their helpful comments that will improve this work.

First of all, it’s not clear which research gap this paper tries to address…

This study is the very first to produce and evaluate geoscience content on TikTok. TikTok and short-form video content has exploded in popularity, and until this point there had been no research on how to capitalize on this success for geoscience communication efforts. We thus fill a crucial research gap related to science communication efforts via short-form video content and provide the first ever analysis of geoscience content on TikTok. There is no way to predict which videos may go ‘viral’ on TikTok (as is the case for any type of viral content). Virality likely requires some type of luck, but also content resonates with the viewer. Our goal with this study was to produce different types of content (different video length, video topic, presentation style, etc.) and evaluate what trends (if any) we can discern in content that had the highest success in terms of reach and engagement.

The authors listed two goals in the manuscript…

We agree with the reviewer that an included discussion of user comments would be beneficial and strengthen our findings. As we discuss in the section on ethical implications, user privacy on TikTok is an important consideration, and thus we will likely not include any specific comments, as no users gave explicit agreement to be part of this study beyond TikTok’s terms of use. However, we can group comments to categories related to statements like, “That was a cool video,” “I learned something new,” “I have a further question related to this video,” “I know the place that they’re talking about here,” etc. to evaluate the type of user engagement the video receives. Our videos received a total of over 3,500 comments during the study period, so we will concentrate on an analysis of comments for our top ten most viewed videos.

Also, the authors may want to do a more thorough literature review about the existing geoscience efforts on TikTok…

There is no other existing literature related to geoscience communication efforts on TikTok, as ours is the very first study to exist. The majority of geoscience content on TikTok is produced by young individuals who have recently or are currently completing bachelor’s or graduate degrees in the geosciences. Content is often split between “meme-style” videos, showing what’s inside a rock when you break it open, and explainer videos of geology topics. Our organizations are pioneering an effort to produce concerted geoscience education videos on TikTok and encourage more scientists and science communicators to do the same. Our study provides the first knowledge base of what types of geoscience communications videos may be successful on TikTok. While there is no existing literature on the topic, there are many museums that now have TikTok accounts (e.g., the American Museum of Natural History), and an analysis of museum-related content would be complementary to this study, but outside the scope of the current study.

We agree that the term “place-based” can be misleading, so we will change this term to “location-based.”

The authors may want to discuss what their hypothesis works…

There is no singular way to predict the success of a video, and unfortunately there will never be a formula to guarantee success with social media content. However, we have been able to observe certain trends both related to the topic of the video and the ‘construction’ of a video that may aid in its success and increase the impact of the communication effort. For example, after making a video talking about the geology of a specific location that got a large amount of views, we made more videos featuring location-based geology that were also largely successful. As the videos were all made during a pilot period, the pilot was focused on producing various different types of content. After full analysis of the videos (this study), that allows us to fully assess patterns and trends in video success, which can then be incorporated into the development of new videos. 

For example, what are the teaching goals of each video? why do some of the videos get so many views? While some others are not…

The teaching goals of each video are essentially to convey geoscience-related information (in our organizations’ case seismology, geodesy, and topography) in a way that is accessible and engaging to the viewer. There unfortunately is no way to specifically discern the exact reason for why a video is successful, as it is likely a combination of factors (the topic + the design/format + the visuals). 

Regarding the number of GMV videos, they are restricted to when magnitude 6+ earthquakes occur, so it is generally not possible to get a large sample size of videos over a short period of time. However, even with a smaller sample size, the GMV videos are the best way to isolate variables,as the video design/format, visuals and description are the same, and observe if there are any other factors that may impact the videos’ success.

As a ‘lecture-style’ video is our own definition and wording, we will include a more thorough explanation of what we mean with this term. (Essentially, like a teacher would ‘lecture’ to a class using a PowerPoint presentation, our ‘lecture-style’ videos feature the host directly talking to the audience like a teacher would, showing visuals and imagery in the background.) 

Moreover, two million views in 4 months with only 48 videos are amazing, but does the views of TikTok comparable to those of YouTube videos?...

Views on TikTok can be difficult to compare to views on YouTube, as TikTok videos are so much shorter than YouTube videos, as noted by the reviewer. For a few of our TikTok videos, we pulled a number of visuals directly from longer videos on the IRIS and UNAVCO YouTube pages, which would be the best comparison of a long-form vs short-form video. However, it is difficult to directly compare the number of views, as some of the YouTube videos have been up for seven years accumulating views and likely have variable spikes in views over time. A future study could perform a more specific direct evaluation of short-form vs. long-form educational geoscience video content.

The best direct comparisons we can provide for TikTok vs. YouTube as platforms are the Terra Explore lidar TikTok videos that were uploaded to the OpenTopography YouTube channel as ‘YouTube Shorts.’ The same exact videos on YouTube received extremely few views, orders of magnitude lower, compared to the videos on TikTok. This observation would suggest that there is a unique opportunity for expanded reach of science communication topics on TikTok. However, that is beyond the scope of this paper.

Furthermore, the work provides some valuable data..

We have since performed more rigorous statistical analyses per the comments of Reviewer #1, which will better aid in the clarification of such statements.

Minor notes:

Line 30-32, 291-299:

There is little existing literature that we are aware of that discusses geoscience communication on YouTube. However, there is now a study by Wang et al. (2022) in Geoscience Communication that does evaluate geoscience videos on YouTube that we will include in a review. (Please do let us know of any other specific studies you may be familiar with.)  While YouTube and TikTok both are video platforms, the short-form, ‘casual’, vertical videos of TikTok and its algorithm-driven nature make them less comparable than at first glance. However, we can include more of a discussion of our organizations’ own efforts on YouTube, which differ quite greatly from the videos produced for TikTok. For example, the OpenTopography YouTube channel primarily features tutorials and webinars geared towards researchers and scientists, rather than direct public outreach efforts. IRIS on their YouTube have videos that range from public outreach (e.g., ‘Women in Geoscience Series’) to educational animations and explainers. UNAVCO’s YouTube channel largely has videos from webinars and short course videos.

Line 60: 

TikTok now does allow users to directly upload a singular 10 minute video file within the app or via desktop, however you can only film videos on TikTok that are 3 minutes in length (which is the primary mode of creation).

Literature Review is not enough…

There currently is no other existing literature related to the geosciences on TikTok, and there are very few other studies that evaluate or discuss science communication on TikTok or short-form video. Our study is the very first to create and evaluate geoscience communication on TikTok, and we are pioneering evaluations and investigations rather than drawing upon an existing knowledge base. The analytical toolkit that we use is more generally related to performance of content on social media. We can include a further discussion of the types of geoscience videos on TikTok (primarily produced by individuals who have recently graduated or are currently in bachelor’s or graduate geoscience programs).

Regarding figure 5…

Shares are not reported as a video view category. There is no way to know how or where a video has been shared (copied the link vs. texted to a friend vs. posted to a social platform, etc.). We hypothesize that video views from shares are counted as a ‘Personal Profile’ view, as videos shared are the direct link to the video. Additionally, a video being shared does not equate to a video view, and there is no way to know whether the video was actually viewed when it was shared.

Line 350-355: 

We have no reason to believe that the gender percentages reported on TikTok are unreliable. While users do not self-contribute this data, TikTok works to discern gender from accounts that the user links (Facebook, Instagram) where users can report gender, or from algorithmic assessment of user behavior, users’ names, etc. (If anything, TikTok is known for over-mining user data). These are the same demographics that are given to multi-million dollar companies that run targeted advertising campaigns on the app, who would rely on robust demographic data. We do note though that the gender % of followers does not account for non-binary individuals, and what is lacking is a more nuanced catagorization of gender.

I personally want to see more data about the timing of geohazard events…

All videos related to geohazards (primarily earthquake and GMV videos) were released the day of or the day after the event. Thus, there is essentially no lag time that can be analyzed.

The demonstration using food could not be the major reason for different views…

What we suggest in the text is that the difference in the videos’ performance is that one video was ~30 s, and the other video was ~1 min. The significantly longer duration of the video is what we hypothesize led to its smaller reach, suggesting that shorter demonstration videos better capture an audience’s attention.

The authors may want to clarify how this work contributes to broader theoretical debates like how geohazards affect or how place-based design affects engagements..

There is an upcoming abstract from the Geological Society of America 2022 meeting that appears relevant to this point: (doi: 10.1130/abs/2022AM-379409) “It is recommended to practitioners for devising pedagogically-sound lessons on any geology/environmental science-related topic to include using as many recent, real-world incident examples as possible and especially relying on controversial, debatable, and sensational sub-topics (within reason).” We can connect our findings to others like these that observed increased student engagement from teaching topics related to geohazards and location-based geology.