the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 15 Mar 2023
Improving the Quality of Education in Water Resources Engineering: A Hybrid Fuzzy-AHP-TOPSIS Method
Abstract. Improving the quality of education in universities can play a prominent role in the development of countries. The purpose of this study is to develop a methodology for assessing the quality of education in Water Resources Engineering, one of the sub-disciplines of Civil Engineering, based on Klein's learning model and using the hybrid fuzzy-AHP-TOPSIS method. Four out of the top ten universities in Iran, including Iran University of Science and Technology (IUST), Amirkabir University of Technology (AUT), Shiraz University (SU), and Khajeh Nasir al-Din Toosi University of Technology (KUT) are considered as case studies. First, the weight coefficients were determined by surveying the students in the fuzzy environment using the AHP method, and then these coefficients were transferred to the TOPSIS environment. Finally, the relative closeness of universities (CC) as a performance evaluation criterion in the form of CC (IUST) = 0.54, CC (AUT) = 0.49, CC (SU) = 0.45, and CC (KUT) = 0.39 were obtained. The sensitivity analysis was performed based on the number and type of Klein's qualitative criteria on the model, and Fourier series expansion curves were used to observe the exact behavior of the model and better compare the results. This model of evaluation can have a considerable influence on the education methods improvement in Civil Engineering departments and related fields.
This preprint has been withdrawn.
-
Withdrawal notice
This preprint has been withdrawn.
-
Preprint
(915 KB)
-
Supplement
(390 KB)
-
This preprint has been withdrawn.
- Preprint
(915 KB) - Metadata XML
-
Supplement
(390 KB) - BibTeX
- EndNote
Interactive discussion
Status: closed
-
CC1: 'Comment on gc-2022-16', Aydin Shishegaran, 11 May 2023
I have two questions.
Q1) Can the method be applied in all decision making problems and engineering fields?
Q2) Which type of problems can be sovled by AHP-TOPSIS?
Citation: https://doi.org/10.5194/gc-2022-16-CC1 -
AC1: 'Reply on CC1', Hossein Hamidifar, 15 May 2023
Thank you very much for your feedback on our manuscript. We appreciate your questions and would like to address them as follows:
Q1) Can the method be applied in all decision-making problems and engineering fields?
Response: Thank you for considering our manuscript. Multi-Criteria Decision-Making (MCDM) methods are widely used in various fields, including engineering. In our study, we employed the Hybrid Fuzzy AHP-TOPSIS method, which is a comprehensive approach applicable to decision-making problems based on the availability of options or different criteria required (Chen et al., 1992). The method involves completing questionnaires based on the opinions of relevant experts in the field. By incorporating mathematical techniques, particularly fuzzy logic, and incorporating expert opinions, we enhance the accuracy of the results obtained. While our method provides a robust framework for decision-making problems, its applicability may depend on the specific context and requirements of the given engineering field.
Q2) Which type of problems can be solved by AHP-TOPSIS?
Response: Generally, multi-criteria decision-making methods such as AHP-TOPSIS can be applied to a wide range of problems where different states with distinct measurable features exist (Bozorg-Haddad et al., 2021). Whenever we are faced with multiple options or alternatives that possess different criteria, these methods can be utilized. The primary objective of AHP-TOPSIS and similar methods is to calculate the weight of each criterion based on its importance, enabling the prioritization of criteria for effective decision-making (Piya et al., 2022). Therefore, these methods significantly assist in decision-making processes across various domains.
References:
Bozorg-Haddad, O., Zolghadr-Asli, B., & Loaiciga, H. A. (2021). A handbook on multi-attribute decision-making methods. John Wiley & Sons.
Chen, S. J., Hwang, C. L., Chen, S. J., & Hwang, C. L. (1992). Fuzzy multiple attribute decision making methods (pp. 289-486). Springer Berlin Heidelberg.
Piya, S., Shamsuzzoha, A., Azizuddin, M., Al-Hinai, N., & Erdebilli, B. (2022). Integrated fuzzy AHP-TOPSIS method to analyze green management practice in hospitality industry in the sultanate of Oman. Sustainability, 14(3), 1118.
Citation: https://doi.org/10.5194/gc-2022-16-AC1
-
AC1: 'Reply on CC1', Hossein Hamidifar, 15 May 2023
-
RC1: 'Comment on gc-2022-16', Anonymous Referee #1, 04 Jun 2023
Review report for the paper “Improving the Quality of Education in Water Resources Engineering: A Hybrid Fuzzy-AHP-TOPSIS Method”.
The applicability of the method. Why do we need AHP application in this study? I did not see the author discussing the reason. Therefore, it is impossible to prove the superiority of this model combination in this article. Need detailed further explanation.
Why did you use AHP method for determining criteria weights? Why not other methods like FUCOM, BWM, DIBR or LBWA etc? You should provide a quality discussion where you will show effectiveness of proposed methodology. Discuss advantages/limitations of these methods (FUCOM, BWM, DIBR or LBWA) and provide superiority of your method.
Why have you used TOPSIS? Why not VIKOR, MARCOS, MABAC or MAIRCA for evaluations? Discuss these methods and their advantages and disadvantages.
Insufficient expression on innovative explanations. Does the practical significance of this innovation exist? There is a lack of comparison with previous studies of the same kind. For this point, the innovativeness of the author's statement needs further explanation.
Indicator issues. Is it appropriate for the author to directly use research results in similar literatures into the research questions of this article? Is there a better reference standard in similar studies? In the subdivision question of this article, do you need to further improve the research results of other scholars in the index design? Please give a reliable argument for the indicator design.
Literature review. Add more recent papers published in last three years. Remove papers published before 2018. Based on the LR you should define the scientific gap. Some recent applications of AHP method are missing like below articles: Tutak, M., & Brodny, J. (2022). Evaluating differences in the Level of Working Conditions between the European Union Member States using TOPSIS method. Decision Making: Applications in Management and Engineering, 5(2), 1-29.; Jagtap, M., & Karande, P. (2023). The m-polar fuzzy set ELECTRE-I with revised Simos’ and AHP weight calculation methods for selection of non-traditional machining processes. Decision Making: Applications in Management and Engineering, 6(1), 240-281.; Sivaprakasam, P., & Angamuthu, M. (2023). Generalized Z-fuzzy soft β-covering based rough matrices and its application to MAGDM problem based on AHP method. Decision Making: Applications in Management and Engineering, 6(1), 134-152. and so on.
There is no result robustness. The author needs to give more detailed data references or results.
Add comparisons with other methods.
The results of the application part of the model need to be rearranged, the readability is too poor, and the graphical results provided can’t make people see the differences under different scene settings.
Add the limitations of proposed methodology.
Citation: https://doi.org/10.5194/gc-2022-16-RC1 -
AC2: 'Reply on RC1', Hossein Hamidifar, 20 Jun 2023
Reviewer #1
Dear Reviewer,
Thank you for your comment. We appreciate your feedback and we have made revisions to address your concerns.
Here are the point-by-point responses:
R1C1: The applicability of the method. Why do we need AHP application in this study? I did not see the author discussing the reason. Therefore, it is impossible to prove the superiority of this model combination in this article. Need detailed further explanation.
Response: Thank you very much for your comment. We agree that a more detailed explanation is needed to clarify the applicability and superiority of the AHP method in this study. Multi-criteria decision-making methods, such as the widely utilized Analytic Hierarchy Process (AHP), have proven to be highly effective in numerous contexts. In the context of evaluating the quality of education in higher education institutions, AHP allows for the determination of criteria weights based on the judgments of experts and stakeholders, and this enables a comprehensive evaluation that takes into account the relative importance of different criteria. By combining AHP with the TOPSIS method, as proposed in this study, we can effectively assess the quality of education by considering both qualitative and quantitative factors.
We have added the following paragraph to the Introduction section of the manuscript:
"Based on the existing evidence (Niu et al., 2019; Muhammad et al., 2020), the growing utilization of the AHP method in educational settings has resulted in the prioritization of more appropriate parameters to improve the quantitative and qualitative aspects of educational institutions and research centres."
R1C2: Why did you use AHP method for determining criteria weights? Why not other methods like FUCOM, BWM, DIBR or LBWA etc? You should provide a quality discussion where you will show effectiveness of proposed methodology. Discuss advantages/limitations of these methods (FUCOM, BWM, DIBR or LBWA) and provide superiority of your method.
R1C3: Why have you used TOPSIS? Why not VIKOR, MARCOS, MABAC or MAIRCA for evaluations? Discuss these methods and their advantages and disadvantages.
Response to R1C2 and R1C3: Our priority in choosing the AHP method is to utilize a fundamental, classical, and widely popular approach for determining the weight coefficients of criteria. The other methods you mentioned are also applicable, although some may have less popularity or are used for specific purposes. Two additional advantages of AHP as a criteria weighting method are its ability to assess the consistency and inconsistency of decisions made and the ease of conducting sensitivity analysis among the criteria. In examining the quality of education and learning among students, we have observed relevant studies that have utilized AHP for evaluation. For these reasons, we decided to use the AHP method and expand upon it. Furthermore, in response to reviewer comments, we have added the following text to the introduction section of the manuscript.
"While alternative methods like FUCOM, BWM, DIBR, or LBWA are also employed by researchers, their application is often limited to specific objectives and constraints. However, in this study, the AHP method was preferred due to its widespread popularity among researchers There are various computational methods available for multi-criteria decision-making, each with its own set of advantages and disadvantages. For instance, the VIKOR method is employed in decision-making processes that utilize compromise programming. This method proves useful when the decision maker struggles to determine the superiority of criteria, whether they are proportional or non-proportional. In such cases, the VIKOR method facilitates the identification of a solution that considers multiple criteria (Paradowski and Sałabun, 2021).
Another method, MABAC, has been proposed specifically for ranking research alternatives. It determines rankings based on the distance from the geometric mean of the available options. However, it is important to note that the applicability of this method is limited to specific scenarios, and the resulting rankings may not hold sufficient value for a given problem (Do and Nguyen, 2022).
MAIRCA is yet another multi-criteria decision-making technique that yields rankings for different options upon completion of computations. Inputs for this method include the decision matrix, criterion weights, and types of criteria. Various concepts such as gap, actual weight, and theoretical weight are incorporated into this method and influence the final ranking. Notably, the best option in this technique is determined by the one with the smallest gap. It can be said that this method follows an extensive process to arrive at a solution (Do and Nguyen, 2022).
Among the newer ranking methods for multi-criteria decision-making problems is the MARCOS method. Similar to the TOPSIS method, it focuses on ranking alternatives by constructing a decision matrix. However, this method alone cannot calculate criterion weights and is typically used as a supplementary approach alongside other techniques like AHP (Duc Trung, 2022). Nevertheless, the TOPSIS method was utilized in this particular study. By employing the TOPSIS method, it becomes possible to identify the best possible answers within the range of problem criteria while duly considering the significance of each criterion (Broniewicz and Ogrodnik, 2021; Tutak and Brodny, 2022). The advantages of this method include its ability to handle both positive and negative criteria, accommodate quantitative and qualitative criteria, and convert qualitative criteria into quantitative measures. Additionally, the computational ease associated with this method is another notable feature (Ayan and Abacioglu, 2022; Do and Nguyen, 2022; İnce and Hakan Isik, 2017; Öztaş et al., 2023).”
R1C4: Insufficient expression on innovative explanations. Does the practical significance of this innovation exist? There is a lack of comparison with previous studies of the same kind. For this point, the innovativeness of the author's statement needs further explanation.
Response: In the proposed model, the Klein pattern or Akker pattern has been used to compare nine qualitative education criteria. This pattern encompasses various completed qualitative criteria that other researchers have emphasized. This pattern remains applicable in the modern world today, despite the extensive development of technologies in the field of education (both face-to-face and virtual). For instance, in the past, traditional face-to-face education with conventional physical facilities was prevalent, whereas now virtual education with computer programs and the like is abundantly used, indicating the compatibility of these significant changes with the Klein learning pattern. This research adopts a novel approach, and the authors believe that conducting sensitivity analysis and presenting the obtained results enhance the usefulness of this work. It is hoped that future works will allow for comparing this study with other research in the field of education quality. In accordance with the comment of the reviewer, the last paragraph of the introduction has been rewritten as follows to better highlight the innovation of this study:
"This study aims to demonstrate the application of hybrid multi-criteria decision-making techniques for evaluating the quality of education in higher education institutions. Through sensitivity analysis and charts, it seeks to understand the factors influencing student education and address issues related to educational quality. This provides valuable assistance to researchers and decision-makers, leading to the development of more suitable models for educational development and implementation planning. The study utilizes the Fuzzy-AHP-TOPSIS approach as a powerful MCDM tool for evaluating initial data. This approach improves solution certainty in a fuzzy environment by calculating criteria weights and alternative rankings. The qualitative components of education and learning are analyzed using the Klein model. The proposed methodology offers a comprehensive approach to evaluating teaching and learning quality in specialized fields like WRE. It incorporates diverse criteria, flexibility, high evaluation capability, and appropriate comprehensiveness. The model was implemented in four renowned Iranian universities, evaluating results for eighteen specific cases by varying the number and type of quality criteria. The Fourier series expansion was employed to analyze and investigate the results. Such analyses greatly contribute to better planning and improving the quality of education."
R1C5: Indicator issues. Is it appropriate for the author to directly use research results in similar literatures into the research questions of this article? Is there a better reference standard in similar studies? In the subdivision question of this article, do you need to further improve the research results of other scholars in the index design? Please give a reliable argument for the indicator design.
Response: As mentioned in the previous answers, different qualitative and quantitative models have been proposed for comparing the quality level in universities, and one of the most comprehensive ones that have been revised and enhanced over the years is the Klein or Akker pattern. This educational pattern can encompass various academic institutions and different scientific domains. It is even capable of addressing deficiencies or advancements in educational facilities, making it inclusive of all aspects. Naturally, in this article, we have proposed this pattern as an efficient and comprehensive model. In response to the reviewer's comment, the Conclusion section has been completely rewritten.
R1C6: Literature review. Add more recent papers published in the last three years. Remove papers published before 2018. Based on the LR you should define the scientific gap. Some recent applications of the AHP method are missing, like the articles mentioned: Tutak, M., & Brodny, J. (2022). Evaluating differences in the Level of Working Conditions between the European Union Member States using the TOPSIS method. Decision Making: Applications in Management and Engineering, 5(2), 1-29.; Jagtap, M., & Karande, P. (2023). The m-polar fuzzy set ELECTRE-I with revised Simos’ and AHP weight calculation methods for the selection of non-traditional machining processes. Decision Making: Applications in Management and Engineering, 6(1), 240-281.; Sivaprakasam, P., & Angamuthu, M. (2023). Generalized Z-fuzzy soft β-covering based rough matrices and its application to MAGDM problem based on AHP method. Decision Making: Applications in Management and Engineering, 6(1), 134-152. and so on. There is no result robustness. The author needs to give more detailed data references or results. Add comparisons with other methods. The results of the application part of the model need to be rearranged; the readability is too poor, and the graphical results provided can't make people see the differences under different scene settings. Add the limitations of the proposed methodology.
Response: Thank you for your valuable feedback. We have revised the literature review section by including more recent papers published in the last three years and removing some papers published before 2018. Additionally, we have incorporated the papers you mentioned as references to provide a more comprehensive overview of recent applications of the MCDM methods. We acknowledge the importance of result robustness and have included more detailed data references to enhance the clarity and reliability of our findings. Furthermore, the conclusion part has been rearranged to improve readability, and the graphical results have been revised to better illustrate the differences under different scenario settings.
References
Ayan, B., Abacioglu, S., 2022. Bibliometric Analysis of the MCDM Methods in the Last Decade: WASPAS, MABAC, EDAS, CODAS, COCOSO, and MARCOS. Int. J. Bus. Econ. Stud. 4, 65–85. https://doi.org/10.54821/uiecd.1183443
Broniewicz, E., Ogrodnik, K., 2021. A Comparative Evaluation of Multi-Criteria Analysis Methods for Sustainable Transport. Energies 14, 5100. https://doi.org/10.3390/en14165100
Do, D.T., Nguyen, N.-T., 2022. Applying Cocoso, Mabac, Mairca, Eamr, Topsis and Weight Determination Methods for Multi-Criteria Decision Making in Hole Turning Process. Strojnícky časopis - J. Mech. Eng. 72, 15–40. https://doi.org/10.2478/scjme-2022-0014
Duc Trung, D., 2022. Multi-criteria decision making under the MARCOS method and the weighting methods: applied to milling, grinding and turning processes. Manuf. Rev. 9, 3. https://doi.org/10.1051/mfreview/2022003
İnce, M., Hakan Isik, A., 2017. AHP-TOPSIS Method for Learning Object Metadata Evaluation Travelling Salesman Problem View project Development Of Integrated Quality Control System For Production Defect Detection By Artificial Vision In Industrial Ceramic Tile Production View project. Int. J. Inf. Educ. Technol. 7, 884–887. https://doi.org/10.18178/ijiet.2017.7.12.989
Muhammad, A., Shaikh, A., Naveed, Q.N., Qureshi, M.R.N., 2020. Factors affecting academic Integrity in E-Learning of Saudi arabian Universities. an Investigation Using Delphi and AHP. IEEE Access 8, 16259–16268. https://doi.org/10.1109/ACCESS.2020.2967499
Niu, B., Liu, Q., Chen, Y., 2019. Research on the university innovation and entrepreneurship education comprehensive evaluation based on AHP method. Int. J. Inf. Educ. Technol. 9, 623–628. https://doi.org/10.18178/IJIET.2019.9.9.1278
Öztaş, T., Aytaç Adalı, E., Tuş, A., Öztaş, G.Z., 2023. Ranking Green Universities from MCDM Perspective: MABAC with Gini Coefficient-based Weighting Method. Process Integr. Optim. Sustain. 7, 163–175. https://doi.org/10.1007/s41660-022-00281-z
Paradowski, B., Sałabun, W., 2021. Are the results of MCDA methods reliable? Selection of materials for thermal energy storage. Procedia Comput. Sci. 192, 1313–1322. https://doi.org/10.1016/j.procs.2021.08.135
Tutak, M., Brodny, J., 2022. Evaluating differences in the Level of Working Conditions between the European Union Member States using TOPSIS method. Decis. Mak. Appl. Manag. Eng. 5, 1–29. https://doi.org/10.31181/dmame0305102022t
Citation: https://doi.org/10.5194/gc-2022-16-AC2
-
AC2: 'Reply on RC1', Hossein Hamidifar, 20 Jun 2023
-
RC2: 'Comment on gc-2022-16', Sebastian G. Mutz, 08 Jun 2023
Disclosure of Expertise
I am familiar with many theoretical aspects of decision making and statistics. However, I have not applied the presented methods or related methods in this type of study. Therefore, I cannot comment on all aspects of this study. My comments and suggestions are written as someone who is interested in exploring the use and potential of such methods in an education setting. Please keep these points in mind when reading my comments.
Summary
The manuscript by Ghorbani et el. describes the application of a hybrid decision-making approach to improve the quality of higher education. Data for nine criteria (or qualitative components relevant to learning) were collected for the application of this approach at four different universities in Iran. In my understanding of the method and study, the approach is merited and can potentially help with informed multi-criteria decision making to improve the quality of education at universities. The manuscript is generally well written, but I do have a few bigger concerns about (and suggestions for) this study/manuscript. Nevertheless, I am confident that the authors will be able to address those, and I look forward to the revised manuscript.
General Comments
1. The study should be clearer on its goal and novelty. In my understanding of the manuscript, I see the novelty in the study primarily in the introduction of the fuzzy aspect and hybrid approach for this particular type of problem. The introduction is well written and introduces the backgrounds nicely. However, toward the end of it, it should be clearer on (a) the novelty of the study, and (b) the need/justification for the proposed method (specifically the fuzzy aspect). In other words, what problems is the study (and novel part of the method) addressing that others did not? I do think the authors could make a very good case for a push toward fuzzy approaches, for example. See specific comments for more details.
2. Generally, I advise that parts of the manuscript are rewritten to allow non-specialists in the specific methods applied here to easily follow the manuscript. These changes would include the addition of brief conceptual descriptions of the methods before the study’s specific applications are detailed. See specific comments for more details.
3. Figures and Tables: I think more informative figures would help convey the work presented here. It is difficult to understand what information figure 2 is meant to convey, for example. Furthermore, it can be tricky to get an overview of the results from just the tables. If the authors can think of a way to visually represent the information, I encourage them to do so. The tables that contain the actual values could then serve as a supplement for readers who wish to have a closer look.
4. Some of the conclusions lack evidence. The benefits of this method (over others) has not been clearly demonstrated or argued.
Specific Comments
L21-22: Here, the authors jump right into weight coefficients and transferring these for the hybrid approach. Those working with AHP may immediately know what is meant. However, I suggest rewriting parts of the abstract to account for a lack of familiarity with the methods among interested readers. I have some knowledge of the mathematical/algorithmic sides of decision making, but it wasn’t clear to me what the weight coefficients referred to until reading the manuscript. Not writing the abstract with the assumption of familiarity with AHP & TOPSIS may also help potential users of the method find an easier entry to it.
L23: The term “relative closeness” in this context also needs a brief explanation. It is a performance criterion for what specifically? Maybe the authors can come up with a way to make this more understandable by explaining it is the closeness to an ideal solution (without getting into too much detail about the method).
L59: Do you mean “pairwise comparison of alternatives”?
L61: Remove underscore after (2010).
L71-76: “[...] data is qualitative, the number of data points is insufficient, the data is insufficiently accurate, or the data is derived from unknown sources” – all these just describe sources of uncertainty. The involvement of “people’s emotions” (which would result in subjective/Bayesian type probabilities/uncertainties) is another example. Before listing these examples, I advise the authors to be a little clearer on what fuzzy set theory/fuzzy mathematics is and what its general strengths are, so the readers can better understand the merits of a fuzzy approach. In contrast to boolean/binary classification or attribution, it deals in degrees (or probability) of attribution. The more general argument can be made that as soon as we have any notable uncertainty in the attribution to any element to any class/group/cluster, a fuzzy approach may be merited and ought to be considered at least. Since the listed goal of the paper is model development, I see some of the novelty of this study in (a) its specific application, and (b) introducing the fuzzy aspects/pushing for the more frequent use of a fuzzy approach to MCDM. Therefore, that aspect should be expanded on a little.
L89: Very minor point: I think the “fame” of the universities (or the status of simply being “well known”, as described later in the text) is less relevant than their good reputation that is a result of their high rankings. I recommend highlighting that (as done in the abstract) instead of their fame.
L95: Klein’s model should be introduced conceptually somewhere. In the introduction, the authors touch on it, but focus more on where it is employed rather than what the model is. I advise the authors to include this somewhere, since they refer to it a lot.
L98: “These elements were researched” is rather vague. What was researched about the elements? I suggest being more specific here to correctly set up the readers expectation of the manuscript.
L101: Please already state the sample size here (i.e., number of WRE students used in the study).
L102: What did those questionnaires, interviews and surveys actually involve? (The appendix should be referenced here and the general nature of the questions should be described). Furthermore, “to collect data” is rather vague. What type of data/information was gathered? Remind the reader also of the goal behind collecting it.
L105: Before jumping into the specific application of the AHP method, it would be most helpful for readers to get a brief, conceptual explanation of the method. I had come across an application of AHP before, yet I struggled to understand everything involved in steps 1-3. Additionally, a justification for the use of the method should be given.
107: “[...] should perform well when comparing all the criteria in a separate pairwise way, […]” is a good example of step descriptions that will likely confuse (a) readers who have not used or read about the AHP method in a while, and (b) readers who are completely unfamiliar with the method and read the manuscript to learn about new criteria ranking and decision making methods for higher education (see L105 comment).
L114-115: (a) “Fuzzy expansion relationships” and (b) where the “weight matrix” fits into the method need to be explained (see L105 comment). Generally, I advise the authors to explain how exactly fuzzy concepts were introduced to the method.
L119-140: I see a similar problem here as in the AHP section. The authors explain more of TOPSIS than AHP, but the problem of lacking explanations persists. TOPSIS needs to be conceptually explained and its use in this study should be justified. What are its weaknesses and strengths (e.g., it is relatively easy to understand)? Why do these method characteristics make it suitable for this particular problem?
L119: I think this sentence can be quite confusing to many. (It only made sense to me, since I work a lot with different types of abstract distances). It’s an example of how the lack of a description (see previous comment) may confuse readers. What type of distance is it? How is the distance between the ideal solution and alternatives being calculated? Before even getting into this, I also advise the authors to explain that an ideal solution is defined (that is not considered fully achievable), and that the goal is to find the closest possible “match”.
L145: I think a figure to help with the explanations above will be extremely useful. However, I do not really understand this particular figure (2). It’s unclear what it is meant to convey. I suggest the authors revise it completely and supplement it with a more informative caption.
L157: Please list the nine qualitative educational characteristics again here, in the table caption and figures 3-4. Since the numbers are referenced a lot, it would be good to have a reference table that the reader can use to look these up while going through the results section.
L233-236: About the Fourier series functions: (a) How does it allow you to determine significance and what type of significance do the authors refer to? (b) While the explanation of a fourier series is correct, I suggest adding a more intuitive explanation for different types of readers.
Figures 3-4: These figures constitute the key results of the study. Once readers understand the methods and calculated metrics, these are informative and easy to understand. I have 2 suggestions for these figures, however: (a) The legend is tricky to read; I advise the use of larger labels. (b) I do not understand the added value of adding the curves to figures 3 and 4. What is gained from the Fourier curves here?
L279: “[…] may better evaluate the quality of education […]”. This statement needs evidence through comparisons. I can see how the approach presented here can be very useful and relatively easy to implement. However, it is not clear why it is better than alternatives. This should be elaborated on through comparisons, evidence and good general arguments.
L289-291: I would welcome the introduction and greater establishment of such methods in other fields, universities and countries. I would therefore be happy to see the suggested changes implemented. They may may help with that exactly.
Citation: https://doi.org/10.5194/gc-2022-16-RC2 -
AC3: 'Reply on RC2', Hossein Hamidifar, 20 Jun 2023
Reviewer #2
Dear Reviewer,
Thank you for your comment. We appreciate your feedback and have made revisions to address your concerns.
Here are the point-by-point responses:
R2C1: The study should be clearer on its goal and novelty. In my understanding of the manuscript, I see the novelty in the study primarily in the introduction of the fuzzy aspect and hybrid approach for this particular type of problem. The introduction is well written and introduces the backgrounds nicely. However, toward the end of it, it should be clearer on (a) the novelty of the study, and (b) the need/justification for the proposed method (specifically the fuzzy aspect). In other words, what problems is the study (and novel part of the method) addressing that others did not? I do think the authors could make a very good case for a push toward fuzzy approaches, for example. See specific comments for more details.
Response: Based on your feedback, we have revised the introduction to provide a clearer explanation of the novelty of the study and the justification for the proposed method. We have emphasized the novelty of introducing the fuzzy aspect and hybrid approach for addressing the specific problem at hand. Additionally, we have highlighted the need for fuzzy approaches in multi-criteria decision-making and the potential benefits they offer compared to traditional methods. This clarification aims to demonstrate the unique contribution of the study and the specific problems it addresses that others have not sufficiently tackled. According to the reviewer’s comment, the following paragraphs have been added to the text:
“Based on the existing evidence (Niu et al., 2019; Muhammad et al., 2020), the growing utilization of the AHP method in educational settings has resulted in the prioritization of more appropriate parameters to improve the quantitative and qualitative aspects of educational institutions and research centres. While alternative methods like FUCOM, BWM, DIBR, or LBWA are also employed by researchers, their application is often limited to specific objectives and constraints. However, in this study, the AHP method was preferred due to its widespread popularity among researchers (Ayough et al., 2023; Chen and Luo, 2023; Jagtap and Karande, 2023; Naz et al., 2023; Sivaprakasam and Angamuthu, 2023).
There are various computational methods available for multi-criteria decision-making, each with its own set of advantages and disadvantages. For instance, the VIKOR method is employed in decision-making processes that utilize compromise programming. This method proves useful when the decision maker struggles to determine the superiority of criteria, whether they are proportional or non-proportional. In such cases, the VIKOR method facilitates the identification of a solution that considers multiple criteria (Paradowski and Sałabun, 2021).
Another method, MABAC, has been proposed specifically for ranking research alternatives. It determines rankings based on the distance from the geometric mean of the available options. However, it is important to note that the applicability of this method is limited to specific scenarios, and the resulting rankings may not hold sufficient value for a given problem (Do and Nguyen, 2022).
MAIRCA is yet another multi-criteria decision-making technique that yields rankings for different options upon completion of computations. Inputs for this method include the decision matrix, criterion weights, and types of criteria. Various concepts such as gap, actual weight, and theoretical weight are incorporated into this method and influence the final ranking. Notably, the best option in this technique is determined by the one with the smallest gap. It can be said that this method follows an extensive process to arrive at a solution (Do and Nguyen, 2022).
Among the newer ranking methods for multi-criteria decision-making problems is the MARCOS method. Similar to the TOPSIS method, it focuses on ranking alternatives by constructing a decision matrix. However, this method alone cannot calculate criterion weights and is typically used as a supplementary approach alongside other techniques like AHP (Duc Trung, 2022). Nevertheless, the TOPSIS method was utilized in this particular study. By employing the TOPSIS method, it becomes possible to identify the best possible answers within the range of problem criteria while duly considering the significance of each criterion (Broniewicz and Ogrodnik, 2021; Tutak and Brodny, 2022). The advantages of this method include its ability to handle both positive and negative criteria, accommodate quantitative and qualitative criteria, and convert qualitative criteria into quantitative measures. Additionally, the computational ease associated with this method is another notable feature (Ayan and Abacioglu, 2022; Do and Nguyen, 2022; İnce and Hakan Isik, 2017; Öztaş et al., 2023).”
“This study aims to demonstrate the application of hybrid multi-criteria decision-making techniques for evaluating the quality of education in higher education institutions. Through sensitivity analysis and charts, it seeks to understand the factors influencing student education and address issues related to educational quality. This provides valuable assistance to researchers and decision-makers, leading to the development of more suitable models for educational development and implementation planning. The study utilizes the Fuzzy-AHP-TOPSIS approach as a powerful MCDM tool for evaluating initial data. This approach improves solution certainty in a fuzzy environment by calculating criteria weights and alternative rankings. The qualitative components of education and learning are analyzed using the Klein model. The proposed methodology offers a comprehensive approach to evaluating teaching and learning quality in specialized fields like WRE. It incorporates diverse criteria, flexibility, high evaluation capability, and appropriate comprehensiveness. The model was implemented in four renowned Iranian universities, evaluating results for eighteen specific cases by varying the number and type of quality criteria. The Fourier series expansion was employed to analyze and investigate the results. Such analyses greatly contribute to better planning and improving the quality of education.”
R2C2: Generally, I advise that parts of the manuscript are rewritten to allow non-specialists in the specific methods applied here to easily follow the manuscript. These changes would include the addition of brief conceptual descriptions of the methods before the study’s specific applications are detailed. See specific comments for more details.
Response: We have considered your suggestion and have added brief conceptual descriptions of the methods employed in the study. These descriptions are provided before the detailed applications of the methods, allowing non-specialists to understand the fundamental concepts and follow the manuscript more easily. By including these explanations, we aim to make the study more accessible to readers who may not be familiar with the specific methods utilized.
R2C3: Figures and Tables: I think more informative figures would help convey the work presented here. It is difficult to understand what information figure 2 is meant to convey, for example. Furthermore, it can be tricky to get an overview of the results from just the tables. If the authors can think of a way to visually represent the information, I encourage them to do so. The tables that contain the actual values could then serve as a supplement for readers who wish to have a closer look.
Response: Thank you for your valuable input regarding the presentation of our work. We have taken your suggestion into consideration and made revisions to Figure 2 to ensure it conveys the information more clearly. While we understand the desire for visual representations, we believe that presenting the results in tables offers several advantages over figures. Firstly, tables provide a concise and organized format for presenting detailed numerical data. They allow for easy comparison and reference, as readers can quickly locate specific values or compare different variables. This structured presentation enhances the clarity and accuracy of the information being conveyed. Additionally, tables provide a comprehensive overview of the results, allowing readers to grasp the complete set of findings at a glance. In contrast, figures may be limited in their capacity to present all relevant details, potentially leading to oversimplification or loss of critical information.
R2C4: Some of the conclusions lack evidence. The benefits of this method (over others) has not been clearly demonstrated or argued.
Response: We acknowledge your comment regarding the need for more evidence to support the conclusions. To address this concern, we have expanded the discussion in the conclusion section, providing more substantial evidence and arguments. We have specifically emphasized the benefits of the proposed method over other approaches, highlighting its ability to handle both positive and negative criteria, accommodate quantitative and qualitative measures, and convert qualitative criteria into quantitative evaluations. These revisions aim to strengthen the conclusions and provide a clearer demonstration of the advantages of the proposed methodology.
R2C5: L21-22: Here, the authors jump right into weight coefficients and transferring these for the hybrid approach. Those working with AHP may immediately know what is meant. However, I suggest rewriting parts of the abstract to account for a lack of familiarity with the methods among interested readers. I have some knowledge of the mathematical/algorithmic sides of decision making, but it wasn’t clear to me what the weight coefficients referred to until reading the manuscript. Not writing the abstract with the assumption of familiarity with AHP & TOPSIS may also help potential users of the method find an easier entry to it.
Response: We have carefully reviewed your suggestion and have revised the abstract accordingly. We have added a brief explanation of weight coefficients and their transfer in the fuzzy environment using the AHP method. This clarification aims to ensure that interested readers, including those unfamiliar with AHP and TOPSIS, can have an easier entry into the study and comprehend the key concepts involved. The following sentences have been added to the Abstract section:
“First, the weight coefficients were determined by surveying the students in a fuzzy environment using the AHP method. This step prioritizes the criteria of the problem based on the experts' perspective. Second, these coefficients were then transferred to the TOPSIS environment, where the previously prioritized criteria are utilized to select the ideal solution to the problem.”
R2C6: L23: The term “relative closeness” in this context also needs a brief explanation. It is a performance criterion for what specifically? Maybe the authors can come up with a way to make this more understandable by explaining it is the closeness to an ideal solution (without getting into too much detail about the method).
Response: Thank you for pointing out the need for a brief explanation of the term "relative closeness." We have addressed this concern by adding an explanation in Step 8 of the methodology section, specifying that it represents the closeness to an ideal solution. This addition aims to provide a clearer understanding of the concept without delving into excessive technical details.
R2C7: L59: Do you mean “pairwise comparison of alternatives”?
Response: We appreciate your observation, and we have made the necessary revision. The term text has been revised in the text as suggested.
R2C8: L61: Remove underscore after (2010).
Response: The underscore after (2010) has been removed.
R2C9: L71-76: “[...] data is qualitative, the number of data points is insufficient, the data is insufficiently accurate, or the data is derived from unknown sources” – all these just describe sources of uncertainty. The involvement of “people’s emotions” (which would result in subjective/Bayesian type probabilities/uncertainties) is another example. Before listing these examples, I advise the authors to be a little clearer on what fuzzy set theory/fuzzy mathematics is and what its general strengths are, so the readers can better understand the merits of a fuzzy approach. In contrast to boolean/binary classification or attribution, it deals in degrees (or probability) of attribution. The more general argument can be made that as soon as we have any notable uncertainty in the attribution to any element to any class/group/cluster, a fuzzy approach may be merited and ought to be considered at least. Since the listed goal of the paper is model development, I see some of the novelty of this study in (a) its specific application, and (b) introducing the fuzzy aspects/pushing for the more frequent use of a fuzzy approach to MCDM. Therefore, that aspect should be expanded on a little.
Response: We have taken your advice into consideration and have revised the explanation of sources of uncertainty. We have provided a more comprehensive explanation of the fuzzy set theory and its general strengths, emphasizing its ability to handle degrees of uncertainty and attribute elements to classes or clusters based on probabilities or degrees of membership. Additionally, we have expanded on the novelty of the study in terms of its specific application and the introduction of fuzzy aspects in multi-criteria decision-making. These revisions aim to provide a more thorough understanding of the merits of the fuzzy approach and the unique contributions of the study.
R2C10: L89: Very minor point: I think the “fame” of the universities (or the status of simply being “well known”, as described later in the text) is less relevant than their good reputation that is a result of their high rankings. I recommend highlighting that (as done in the abstract) instead of their fame.
Response: We agree with your point and have revised the text accordingly. The emphasis has been shifted from the fame of the universities to their good reputation resulting from high rankings. The revised text now highlights the universities' reputation as a reflection of their quality and ranking rather than their mere fame. the text has been revised as below:
“The model was implemented in four out of the top ten universities in Iran, evaluating results for eighteen specific cases by varying the number and type of quality criteria.”
R2C11: L95: Klein’s model should be introduced conceptually somewhere. In the introduction, the authors touch on it, but focus more on where it is employed rather than what the model is. I advise the authors to include this somewhere, since they refer to it a lot.
Response: Thank you for your comment. We understand the importance of introducing Klein's model conceptually to provide a better understanding of its relevance to our study. As our focus is on utilizing the findings of Klein's research and maintaining brevity in writing, we acknowledge that more information about the model would be beneficial. Because of the limitations regarding the number of manuscript pages, we have provided references for readers who wish to explore the model further.
R2C12: L98: “These elements were researched” is rather vague. What was researched about the elements? I suggest being more specific here to correctly set up the readers expectation of the manuscript.
Response: We appreciate your suggestion to be more specific about what was researched regarding the elements. In response, we have revised the text as follows: "All the qualitative criteria proposed by Klein were examined…"
R2C13: L101: Please already state the sample size here (i.e., number of WRE students used in the study).
Response: Thank you for pointing out the need to state the sample size explicitly. We have revised the text as follows: "This study's population consists of 112 WRE students from a diverse range of universities."
R2C14: L102: What did those questionnaires, interviews and surveys actually involve? (The appendix should be referenced here and the general nature of the questions should be described). Furthermore, “to collect data” is rather vague. What type of data/information was gathered? Remind the reader also of the goal behind collecting it.
Response: We acknowledge your comment and agree that providing more information about the questionnaires, interviews, and surveys would enhance the reader's understanding. We have included a reference to the appendix where the questionnaire is provided and have described the nature of the questions in the text. Additionally, we have clarified the goal behind collecting the data, emphasizing that it aims to determine the relative importance of the desired criteria based on participants' specific university conditions. The revised text now reads as follows: "Considering the nature of the AHP method, which is based on pairwise comparisons, the prioritization of alternatives (i.e., the four mentioned universities) and criteria (i.e., the nine qualitative criteria introduced by Klein) has been provided to participants in the form of questionnaires. The participants are asked to complete the questionnaires based on the specific conditions of their own university to determine the relative importance of the desired criteria. The questionnaire (Appendix 1) includes items such as the content of course materials, learning activities, the role of instructors in learning, educational materials and resources, groupings and collective participation, suitable learning environment and time, and assessment."
R2C15 and R2C16: L105: Before jumping into the specific application of the AHP method, it would be most helpful for readers to get a brief, conceptual explanation of the method. I had come across an application of AHP before, yet I struggled to understand everything involved in steps 1-3. Additionally, a justification for the use of the method should be given.
and 107: “[...] should perform well when comparing all the criteria in a separate pairwise way, […]” is a good example of step descriptions that will likely confuse (a) readers who have not used or read about the AHP method in a while, and (b) readers who are completely unfamiliar with the method and read the manuscript to learn about new criteria ranking and decision making methods for higher education (see L105 comment).
Response: We appreciate your feedback. We have provided a justification for using the method, emphasizing its widespread acceptance and effectiveness in ranking criteria and making decisions in various domains. These revisions aim to provide readers, including those unfamiliar with AHP, with a better understanding of the method and its relevance to the study.
R2C17: L114-115: (a) “Fuzzy expansion relationships” and (b) where the “weight matrix” fits into the method need to be explained (see L105 comment). Generally, I advise the authors to explain how exactly fuzzy concepts were introduced to the method.
Response: Thank you for your comment regarding the explanation of fuzzy concepts in our study. We acknowledge the importance of providing clarity on how fuzzy concepts were introduced to the AHP method. While various methods have been proposed by researchers for converting real data into fuzzy data, we utilized simple techniques that have been previously presented in studies by Sirisawat and Kiatcharoenpol (2018) and Zavadskas et al. (2020). We have included references to these studies in the text to allow interested readers to explore the details further. We understand that due to the length constraints of the article, it is not possible to present all the specific details in the text. However, we believe that these references will provide readers with valuable information on the implementation of fuzzy concepts in the AHP method.
R2C18: L119-140: I see a similar problem here as in the AHP section. The authors explain more of TOPSIS than AHP, but the problem of lacking explanations persists. TOPSIS needs to be conceptually explained and its use in this study should be justified. What are its weaknesses and strengths (e.g., it is relatively easy to understand)? Why do these method characteristics make it suitable for this particular problem?
Response: Thank you for your valuable feedback regarding the explanation of the TOPSIS method in our study. We recognize the need for a more comprehensive conceptual explanation of TOPSIS and a justification for its use in this research. In response to your comment and taking into account the length constraints of the manuscript, we have expanded the text to provide more information about the different Multi-Criteria Decision Making (MCDM) methods, including TOPSIS. The following paragraphs have been added to the text:
“Based on the existing evidence (Niu et al., 2019; Muhammad et al., 2020), the growing utilization of the AHP method in educational settings has resulted in the prioritization of more appropriate parameters to improve the quantitative and qualitative aspects of educational institutions and research centres. While alternative methods like FUCOM, BWM, DIBR, or LBWA are also employed by researchers, their application is often limited to specific objectives and constraints. However, in this study, the AHP method was preferred due to its widespread popularity among researchers (Ayough et al., 2023; Chen and Luo, 2023; Jagtap and Karande, 2023; Naz et al., 2023; Sivaprakasam and Angamuthu, 2023).
There are various computational methods available for multi-criteria decision-making, each with its own set of advantages and disadvantages. For instance, the VIKOR method is employed in decision-making processes that utilize compromise programming. This method proves useful when the decision maker struggles to determine the superiority of criteria, whether they are proportional or non-proportional. In such cases, the VIKOR method facilitates the identification of a solution that considers multiple criteria (Paradowski and Sałabun, 2021).
Another method, MABAC, has been proposed specifically for ranking research alternatives. It determines rankings based on the distance from the geometric mean of the available options. However, it is important to note that the applicability of this method is limited to specific scenarios, and the resulting rankings may not hold sufficient value for a given problem (Do and Nguyen, 2022).
MAIRCA is yet another multi-criteria decision-making technique that yields rankings for different options upon completion of computations. Inputs for this method include the decision matrix, criterion weights, and types of criteria. Various concepts such as gap, actual weight, and theoretical weight are incorporated into this method and influence the final ranking. Notably, the best option in this technique is determined by the one with the smallest gap. It can be said that this method follows an extensive process to arrive at a solution (Do and Nguyen, 2022).
Among the newer ranking methods for multi-criteria decision-making problems is the MARCOS method. Similar to the TOPSIS method, it focuses on ranking alternatives by constructing a decision matrix. However, this method alone cannot calculate criterion weights and is typically used as a supplementary approach alongside other techniques like AHP (Duc Trung, 2022). Nevertheless, the TOPSIS method was utilized in this particular study. By employing the TOPSIS method, it becomes possible to identify the best possible answers within the range of problem criteria while duly considering the significance of each criterion (Broniewicz and Ogrodnik, 2021; Tutak and Brodny, 2022). The advantages of this method include its ability to handle both positive and negative criteria, accommodate quantitative and qualitative criteria, and convert qualitative criteria into quantitative measures. Additionally, the computational ease associated with this method is another notable feature (Ayan and Abacioglu, 2022; Do and Nguyen, 2022; İnce and Hakan Isik, 2017; Öztaş et al., 2023).”
R2C19: L119: I think this sentence can be quite confusing to many. (It only made sense to me, since I work a lot with different types of abstract distances). It’s an example of how the lack of a description (see previous comment) may confuse readers. What type of distance is it? How is the distance between the ideal solution and alternatives being calculated? Before even getting into this, I also advise the authors to explain that an ideal solution is defined (that is not considered fully achievable), and that the goal is to find the closest possible “match”.
Response: Thank you for your feedback on the clarity of the sentence regarding the relative closeness coefficient. We understand that the lack of a description may confuse readers who are not familiar with this concept. To address this concern and provide a better understanding, we have revised the sentence as follows:
“The relative closeness coefficient ( ), that represents the closeness to an ideal solution or …”
R2C20: L145: I think a figure to help with the explanations above will be extremely useful. However, I do not really understand this particular figure (2). It’s unclear what it is meant to convey. I suggest the authors revise it completely and supplement it with a more informative caption.
Response: Thank you for your feedback regarding Figure 2. We agree that a visual aid can greatly enhance the understanding of the concepts explained in the manuscript. Based on your suggestion, we have completely revised Figure 2 to make it more informative and helpful. The new figure provides a clear representation of the concepts discussed in the text.
R2C21: L157: Please list the nine qualitative educational characteristics again here, in the table caption and figures 3-4. Since the numbers are referenced a lot, it would be good to have a reference table that the reader can use to look these up while going through the results section.
Response: Thank you for your feedback regarding the listing of the nine qualitative educational characteristics. We understand your concern about having a reference table for easy access to the criteria while reading the results section. To address this, all of the nine criteria are listed in Figure 2 and also in Table 1. However, they are not repeated in the text for brevity and the criteria are referenced with their unique number in other figures and tables.
R2C22: L233-236: About the Fourier series functions: (a) How does it allow you to determine significance and what type of significance do the authors refer to? (b) While the explanation of a fourier series is correct, I suggest adding a more intuitive explanation for different types of readers.
Response: The Fourier series functions are commonly used to fit data in various problems. In the current research, these Fourier series functions have been employed based on data fitting, and they have been briefly explained in the article to enhance the understanding of the results. For example please see Line 287-290: “This characteristic of MCDM approaches allows for the application of Fourier series functions (Dyke, 2014) to determine the significance of criteria and evaluate the situation more clearly. A Fourier series is a periodic function extension in terms of an infinite algebraic sum of sines and cosines functions that form a link between these two types of trigonometric functions.”
R2C23: Figures 3-4: These figures constitute the key results of the study. Once readers understand the methods and calculated metrics, these are informative and easy to understand. I have 2 suggestions for these figures, however: (a) The legend is tricky to read; I advise the use of larger labels. (b) I do not understand the added value of adding the curves to figures 3 and 4. What is gained from the Fourier curves here?
Response: The figures have been updated with new legends for improved readability. As for the curves, they have been included to reduce the uncertainties associated with the data points and enhance the precision in identifying the highest and lowest values.
R2C24: L279: “[…] may better evaluate the quality of education […]”. This statement needs evidence through comparisons. I can see how the approach presented here can be very useful and relatively easy to implement. However, it is not clear why it is better than alternatives. This should be elaborated on through comparisons, evidence and good general arguments.
Response: the mentioned sentence has been deleted for the sake of clarity.
R2C25: L289-291: I would welcome the introduction and greater establishment of such methods in other fields, universities and countries. I would therefore be happy to see the suggested changes implemented. They may help with that exactly.
Response: Thank you for your positive feedback and support for the broader adoption of the proposed methods. We share your enthusiasm for introducing these methods in other fields, universities, and countries. By implementing the suggested changes and further developing these models, we aim to contribute to the advancement and wider utilization of this evaluation framework.
References
Ayan, B., Abacioglu, S., 2022. Bibliometric Analysis of the MCDM Methods in the Last Decade: WASPAS, MABAC, EDAS, CODAS, COCOSO, and MARCOS. Int. J. Bus. Econ. Stud. 4, 65–85. https://doi.org/10.54821/uiecd.1183443
Ayough, A., Shargh, S.B., Khorshidvand, B., 2023. A new integrated approach based on base-criterion and utility additive methods and its application to supplier selection problem. Expert Syst. Appl. 221, 119740. https://doi.org/10.1016/J.ESWA.2023.119740
Broniewicz, E., Ogrodnik, K., 2021. A Comparative Evaluation of Multi-Criteria Analysis Methods for Sustainable Transport. Energies 14, 5100. https://doi.org/10.3390/en14165100
Chen, Z., Luo, W., 2023. An integrated interval type-2 fuzzy rough technique for emergency decision making. Appl. Soft Comput. 137, 110150. https://doi.org/10.1016/J.ASOC.2023.110150
Do, D.T., Nguyen, N.-T., 2022. Applying Cocoso, Mabac, Mairca, Eamr, Topsis and Weight Determination Methods for Multi-Criteria Decision Making in Hole Turning Process. Strojnícky časopis - J. Mech. Eng. 72, 15–40. https://doi.org/10.2478/scjme-2022-0014
Duc Trung, D., 2022. Multi-criteria decision making under the MARCOS method and the weighting methods: applied to milling, grinding and turning processes. Manuf. Rev. 9, 3. https://doi.org/10.1051/mfreview/2022003
Dyke, P., 2014. An Introduction to Laplace Transforms and Fourier Series, in: Springer Undergraduate Mathematics Series, Springer Undergraduate Mathematics Series. Springer London, London. https://doi.org/10.1007/978-1-4471-6395-4
İnce, M., Hakan Isik, A., 2017. AHP-TOPSIS Method for Learning Object Metadata Evaluation Travelling Salesman Problem View project Development Of Integrated Quality Control System For Production Defect Detection By Artificial Vision In Industrial Ceramic Tile Production View project. Int. J. Inf. Educ. Technol. 7, 884–887. https://doi.org/10.18178/ijiet.2017.7.12.989
Jagtap, M., Karande, P., 2023. The m-polar fuzzy set ELECTRE-I with revised Simos’ and AHP weight calculation methods for selection of non-traditional machining processes. Decis. Mak. Appl. Manag. Eng. 6, 240–281. https://doi.org/10.31181/DMAME060129022023J
Muhammad, A., Shaikh, A., Naveed, Q.N., Qureshi, M.R.N., 2020. Factors affecting academic Integrity in E-Learning of Saudi arabian Universities. an Investigation Using Delphi and AHP. IEEE Access 8, 16259–16268. https://doi.org/10.1109/ACCESS.2020.2967499
Naz, S., Akram, M., Hassan, M.M. ul, Fatima, A., 2023. A hybrid DEMATEL-TOPSIS approach using 2-tuple linguistic q -rung orthopair fuzzy information and its application in renewable energy resource selection . Int. J. Inf. Technol. Decis. Mak. https://doi.org/10.1142/S0219622023500323
Niu, B., Liu, Q., Chen, Y., 2019. Research on the university innovation and entrepreneurship education comprehensive evaluation based on AHP method. Int. J. Inf. Educ. Technol. 9, 623–628. https://doi.org/10.18178/IJIET.2019.9.9.1278
Öztaş, T., Aytaç Adalı, E., Tuş, A., Öztaş, G.Z., 2023. Ranking Green Universities from MCDM Perspective: MABAC with Gini Coefficient-based Weighting Method. Process Integr. Optim. Sustain. 7, 163–175. https://doi.org/10.1007/s41660-022-00281-z
Paradowski, B., Sałabun, W., 2021. Are the results of MCDA methods reliable? Selection of materials for thermal energy storage. Procedia Comput. Sci. 192, 1313–1322. https://doi.org/10.1016/j.procs.2021.08.135
Sivaprakasam, P., Angamuthu, M., 2023. Generalized Z-fuzzy soft β-covering based rough matrices and its application to MAGDM problem based on AHP method. Decis. Mak. Appl. Manag. Eng. 6, 134–152. https://doi.org/10.31181/dmame04012023p
Tutak, M., Brodny, J., 2022. Evaluating differences in the Level of Working Conditions between the European Union Member States using TOPSIS method. Decis. Mak. Appl. Manag. Eng. 5, 1–29. https://doi.org/10.31181/dmame0305102022t
Citation: https://doi.org/10.5194/gc-2022-16-AC3
-
AC3: 'Reply on RC2', Hossein Hamidifar, 20 Jun 2023
Interactive discussion
Status: closed
-
CC1: 'Comment on gc-2022-16', Aydin Shishegaran, 11 May 2023
I have two questions.
Q1) Can the method be applied in all decision making problems and engineering fields?
Q2) Which type of problems can be sovled by AHP-TOPSIS?
Citation: https://doi.org/10.5194/gc-2022-16-CC1 -
AC1: 'Reply on CC1', Hossein Hamidifar, 15 May 2023
Thank you very much for your feedback on our manuscript. We appreciate your questions and would like to address them as follows:
Q1) Can the method be applied in all decision-making problems and engineering fields?
Response: Thank you for considering our manuscript. Multi-Criteria Decision-Making (MCDM) methods are widely used in various fields, including engineering. In our study, we employed the Hybrid Fuzzy AHP-TOPSIS method, which is a comprehensive approach applicable to decision-making problems based on the availability of options or different criteria required (Chen et al., 1992). The method involves completing questionnaires based on the opinions of relevant experts in the field. By incorporating mathematical techniques, particularly fuzzy logic, and incorporating expert opinions, we enhance the accuracy of the results obtained. While our method provides a robust framework for decision-making problems, its applicability may depend on the specific context and requirements of the given engineering field.
Q2) Which type of problems can be solved by AHP-TOPSIS?
Response: Generally, multi-criteria decision-making methods such as AHP-TOPSIS can be applied to a wide range of problems where different states with distinct measurable features exist (Bozorg-Haddad et al., 2021). Whenever we are faced with multiple options or alternatives that possess different criteria, these methods can be utilized. The primary objective of AHP-TOPSIS and similar methods is to calculate the weight of each criterion based on its importance, enabling the prioritization of criteria for effective decision-making (Piya et al., 2022). Therefore, these methods significantly assist in decision-making processes across various domains.
References:
Bozorg-Haddad, O., Zolghadr-Asli, B., & Loaiciga, H. A. (2021). A handbook on multi-attribute decision-making methods. John Wiley & Sons.
Chen, S. J., Hwang, C. L., Chen, S. J., & Hwang, C. L. (1992). Fuzzy multiple attribute decision making methods (pp. 289-486). Springer Berlin Heidelberg.
Piya, S., Shamsuzzoha, A., Azizuddin, M., Al-Hinai, N., & Erdebilli, B. (2022). Integrated fuzzy AHP-TOPSIS method to analyze green management practice in hospitality industry in the sultanate of Oman. Sustainability, 14(3), 1118.
Citation: https://doi.org/10.5194/gc-2022-16-AC1
-
AC1: 'Reply on CC1', Hossein Hamidifar, 15 May 2023
-
RC1: 'Comment on gc-2022-16', Anonymous Referee #1, 04 Jun 2023
Review report for the paper “Improving the Quality of Education in Water Resources Engineering: A Hybrid Fuzzy-AHP-TOPSIS Method”.
The applicability of the method. Why do we need AHP application in this study? I did not see the author discussing the reason. Therefore, it is impossible to prove the superiority of this model combination in this article. Need detailed further explanation.
Why did you use AHP method for determining criteria weights? Why not other methods like FUCOM, BWM, DIBR or LBWA etc? You should provide a quality discussion where you will show effectiveness of proposed methodology. Discuss advantages/limitations of these methods (FUCOM, BWM, DIBR or LBWA) and provide superiority of your method.
Why have you used TOPSIS? Why not VIKOR, MARCOS, MABAC or MAIRCA for evaluations? Discuss these methods and their advantages and disadvantages.
Insufficient expression on innovative explanations. Does the practical significance of this innovation exist? There is a lack of comparison with previous studies of the same kind. For this point, the innovativeness of the author's statement needs further explanation.
Indicator issues. Is it appropriate for the author to directly use research results in similar literatures into the research questions of this article? Is there a better reference standard in similar studies? In the subdivision question of this article, do you need to further improve the research results of other scholars in the index design? Please give a reliable argument for the indicator design.
Literature review. Add more recent papers published in last three years. Remove papers published before 2018. Based on the LR you should define the scientific gap. Some recent applications of AHP method are missing like below articles: Tutak, M., & Brodny, J. (2022). Evaluating differences in the Level of Working Conditions between the European Union Member States using TOPSIS method. Decision Making: Applications in Management and Engineering, 5(2), 1-29.; Jagtap, M., & Karande, P. (2023). The m-polar fuzzy set ELECTRE-I with revised Simos’ and AHP weight calculation methods for selection of non-traditional machining processes. Decision Making: Applications in Management and Engineering, 6(1), 240-281.; Sivaprakasam, P., & Angamuthu, M. (2023). Generalized Z-fuzzy soft β-covering based rough matrices and its application to MAGDM problem based on AHP method. Decision Making: Applications in Management and Engineering, 6(1), 134-152. and so on.
There is no result robustness. The author needs to give more detailed data references or results.
Add comparisons with other methods.
The results of the application part of the model need to be rearranged, the readability is too poor, and the graphical results provided can’t make people see the differences under different scene settings.
Add the limitations of proposed methodology.
Citation: https://doi.org/10.5194/gc-2022-16-RC1 -
AC2: 'Reply on RC1', Hossein Hamidifar, 20 Jun 2023
Reviewer #1
Dear Reviewer,
Thank you for your comment. We appreciate your feedback and we have made revisions to address your concerns.
Here are the point-by-point responses:
R1C1: The applicability of the method. Why do we need AHP application in this study? I did not see the author discussing the reason. Therefore, it is impossible to prove the superiority of this model combination in this article. Need detailed further explanation.
Response: Thank you very much for your comment. We agree that a more detailed explanation is needed to clarify the applicability and superiority of the AHP method in this study. Multi-criteria decision-making methods, such as the widely utilized Analytic Hierarchy Process (AHP), have proven to be highly effective in numerous contexts. In the context of evaluating the quality of education in higher education institutions, AHP allows for the determination of criteria weights based on the judgments of experts and stakeholders, and this enables a comprehensive evaluation that takes into account the relative importance of different criteria. By combining AHP with the TOPSIS method, as proposed in this study, we can effectively assess the quality of education by considering both qualitative and quantitative factors.
We have added the following paragraph to the Introduction section of the manuscript:
"Based on the existing evidence (Niu et al., 2019; Muhammad et al., 2020), the growing utilization of the AHP method in educational settings has resulted in the prioritization of more appropriate parameters to improve the quantitative and qualitative aspects of educational institutions and research centres."
R1C2: Why did you use AHP method for determining criteria weights? Why not other methods like FUCOM, BWM, DIBR or LBWA etc? You should provide a quality discussion where you will show effectiveness of proposed methodology. Discuss advantages/limitations of these methods (FUCOM, BWM, DIBR or LBWA) and provide superiority of your method.
R1C3: Why have you used TOPSIS? Why not VIKOR, MARCOS, MABAC or MAIRCA for evaluations? Discuss these methods and their advantages and disadvantages.
Response to R1C2 and R1C3: Our priority in choosing the AHP method is to utilize a fundamental, classical, and widely popular approach for determining the weight coefficients of criteria. The other methods you mentioned are also applicable, although some may have less popularity or are used for specific purposes. Two additional advantages of AHP as a criteria weighting method are its ability to assess the consistency and inconsistency of decisions made and the ease of conducting sensitivity analysis among the criteria. In examining the quality of education and learning among students, we have observed relevant studies that have utilized AHP for evaluation. For these reasons, we decided to use the AHP method and expand upon it. Furthermore, in response to reviewer comments, we have added the following text to the introduction section of the manuscript.
"While alternative methods like FUCOM, BWM, DIBR, or LBWA are also employed by researchers, their application is often limited to specific objectives and constraints. However, in this study, the AHP method was preferred due to its widespread popularity among researchers There are various computational methods available for multi-criteria decision-making, each with its own set of advantages and disadvantages. For instance, the VIKOR method is employed in decision-making processes that utilize compromise programming. This method proves useful when the decision maker struggles to determine the superiority of criteria, whether they are proportional or non-proportional. In such cases, the VIKOR method facilitates the identification of a solution that considers multiple criteria (Paradowski and Sałabun, 2021).
Another method, MABAC, has been proposed specifically for ranking research alternatives. It determines rankings based on the distance from the geometric mean of the available options. However, it is important to note that the applicability of this method is limited to specific scenarios, and the resulting rankings may not hold sufficient value for a given problem (Do and Nguyen, 2022).
MAIRCA is yet another multi-criteria decision-making technique that yields rankings for different options upon completion of computations. Inputs for this method include the decision matrix, criterion weights, and types of criteria. Various concepts such as gap, actual weight, and theoretical weight are incorporated into this method and influence the final ranking. Notably, the best option in this technique is determined by the one with the smallest gap. It can be said that this method follows an extensive process to arrive at a solution (Do and Nguyen, 2022).
Among the newer ranking methods for multi-criteria decision-making problems is the MARCOS method. Similar to the TOPSIS method, it focuses on ranking alternatives by constructing a decision matrix. However, this method alone cannot calculate criterion weights and is typically used as a supplementary approach alongside other techniques like AHP (Duc Trung, 2022). Nevertheless, the TOPSIS method was utilized in this particular study. By employing the TOPSIS method, it becomes possible to identify the best possible answers within the range of problem criteria while duly considering the significance of each criterion (Broniewicz and Ogrodnik, 2021; Tutak and Brodny, 2022). The advantages of this method include its ability to handle both positive and negative criteria, accommodate quantitative and qualitative criteria, and convert qualitative criteria into quantitative measures. Additionally, the computational ease associated with this method is another notable feature (Ayan and Abacioglu, 2022; Do and Nguyen, 2022; İnce and Hakan Isik, 2017; Öztaş et al., 2023).”
R1C4: Insufficient expression on innovative explanations. Does the practical significance of this innovation exist? There is a lack of comparison with previous studies of the same kind. For this point, the innovativeness of the author's statement needs further explanation.
Response: In the proposed model, the Klein pattern or Akker pattern has been used to compare nine qualitative education criteria. This pattern encompasses various completed qualitative criteria that other researchers have emphasized. This pattern remains applicable in the modern world today, despite the extensive development of technologies in the field of education (both face-to-face and virtual). For instance, in the past, traditional face-to-face education with conventional physical facilities was prevalent, whereas now virtual education with computer programs and the like is abundantly used, indicating the compatibility of these significant changes with the Klein learning pattern. This research adopts a novel approach, and the authors believe that conducting sensitivity analysis and presenting the obtained results enhance the usefulness of this work. It is hoped that future works will allow for comparing this study with other research in the field of education quality. In accordance with the comment of the reviewer, the last paragraph of the introduction has been rewritten as follows to better highlight the innovation of this study:
"This study aims to demonstrate the application of hybrid multi-criteria decision-making techniques for evaluating the quality of education in higher education institutions. Through sensitivity analysis and charts, it seeks to understand the factors influencing student education and address issues related to educational quality. This provides valuable assistance to researchers and decision-makers, leading to the development of more suitable models for educational development and implementation planning. The study utilizes the Fuzzy-AHP-TOPSIS approach as a powerful MCDM tool for evaluating initial data. This approach improves solution certainty in a fuzzy environment by calculating criteria weights and alternative rankings. The qualitative components of education and learning are analyzed using the Klein model. The proposed methodology offers a comprehensive approach to evaluating teaching and learning quality in specialized fields like WRE. It incorporates diverse criteria, flexibility, high evaluation capability, and appropriate comprehensiveness. The model was implemented in four renowned Iranian universities, evaluating results for eighteen specific cases by varying the number and type of quality criteria. The Fourier series expansion was employed to analyze and investigate the results. Such analyses greatly contribute to better planning and improving the quality of education."
R1C5: Indicator issues. Is it appropriate for the author to directly use research results in similar literatures into the research questions of this article? Is there a better reference standard in similar studies? In the subdivision question of this article, do you need to further improve the research results of other scholars in the index design? Please give a reliable argument for the indicator design.
Response: As mentioned in the previous answers, different qualitative and quantitative models have been proposed for comparing the quality level in universities, and one of the most comprehensive ones that have been revised and enhanced over the years is the Klein or Akker pattern. This educational pattern can encompass various academic institutions and different scientific domains. It is even capable of addressing deficiencies or advancements in educational facilities, making it inclusive of all aspects. Naturally, in this article, we have proposed this pattern as an efficient and comprehensive model. In response to the reviewer's comment, the Conclusion section has been completely rewritten.
R1C6: Literature review. Add more recent papers published in the last three years. Remove papers published before 2018. Based on the LR you should define the scientific gap. Some recent applications of the AHP method are missing, like the articles mentioned: Tutak, M., & Brodny, J. (2022). Evaluating differences in the Level of Working Conditions between the European Union Member States using the TOPSIS method. Decision Making: Applications in Management and Engineering, 5(2), 1-29.; Jagtap, M., & Karande, P. (2023). The m-polar fuzzy set ELECTRE-I with revised Simos’ and AHP weight calculation methods for the selection of non-traditional machining processes. Decision Making: Applications in Management and Engineering, 6(1), 240-281.; Sivaprakasam, P., & Angamuthu, M. (2023). Generalized Z-fuzzy soft β-covering based rough matrices and its application to MAGDM problem based on AHP method. Decision Making: Applications in Management and Engineering, 6(1), 134-152. and so on. There is no result robustness. The author needs to give more detailed data references or results. Add comparisons with other methods. The results of the application part of the model need to be rearranged; the readability is too poor, and the graphical results provided can't make people see the differences under different scene settings. Add the limitations of the proposed methodology.
Response: Thank you for your valuable feedback. We have revised the literature review section by including more recent papers published in the last three years and removing some papers published before 2018. Additionally, we have incorporated the papers you mentioned as references to provide a more comprehensive overview of recent applications of the MCDM methods. We acknowledge the importance of result robustness and have included more detailed data references to enhance the clarity and reliability of our findings. Furthermore, the conclusion part has been rearranged to improve readability, and the graphical results have been revised to better illustrate the differences under different scenario settings.
References
Ayan, B., Abacioglu, S., 2022. Bibliometric Analysis of the MCDM Methods in the Last Decade: WASPAS, MABAC, EDAS, CODAS, COCOSO, and MARCOS. Int. J. Bus. Econ. Stud. 4, 65–85. https://doi.org/10.54821/uiecd.1183443
Broniewicz, E., Ogrodnik, K., 2021. A Comparative Evaluation of Multi-Criteria Analysis Methods for Sustainable Transport. Energies 14, 5100. https://doi.org/10.3390/en14165100
Do, D.T., Nguyen, N.-T., 2022. Applying Cocoso, Mabac, Mairca, Eamr, Topsis and Weight Determination Methods for Multi-Criteria Decision Making in Hole Turning Process. Strojnícky časopis - J. Mech. Eng. 72, 15–40. https://doi.org/10.2478/scjme-2022-0014
Duc Trung, D., 2022. Multi-criteria decision making under the MARCOS method and the weighting methods: applied to milling, grinding and turning processes. Manuf. Rev. 9, 3. https://doi.org/10.1051/mfreview/2022003
İnce, M., Hakan Isik, A., 2017. AHP-TOPSIS Method for Learning Object Metadata Evaluation Travelling Salesman Problem View project Development Of Integrated Quality Control System For Production Defect Detection By Artificial Vision In Industrial Ceramic Tile Production View project. Int. J. Inf. Educ. Technol. 7, 884–887. https://doi.org/10.18178/ijiet.2017.7.12.989
Muhammad, A., Shaikh, A., Naveed, Q.N., Qureshi, M.R.N., 2020. Factors affecting academic Integrity in E-Learning of Saudi arabian Universities. an Investigation Using Delphi and AHP. IEEE Access 8, 16259–16268. https://doi.org/10.1109/ACCESS.2020.2967499
Niu, B., Liu, Q., Chen, Y., 2019. Research on the university innovation and entrepreneurship education comprehensive evaluation based on AHP method. Int. J. Inf. Educ. Technol. 9, 623–628. https://doi.org/10.18178/IJIET.2019.9.9.1278
Öztaş, T., Aytaç Adalı, E., Tuş, A., Öztaş, G.Z., 2023. Ranking Green Universities from MCDM Perspective: MABAC with Gini Coefficient-based Weighting Method. Process Integr. Optim. Sustain. 7, 163–175. https://doi.org/10.1007/s41660-022-00281-z
Paradowski, B., Sałabun, W., 2021. Are the results of MCDA methods reliable? Selection of materials for thermal energy storage. Procedia Comput. Sci. 192, 1313–1322. https://doi.org/10.1016/j.procs.2021.08.135
Tutak, M., Brodny, J., 2022. Evaluating differences in the Level of Working Conditions between the European Union Member States using TOPSIS method. Decis. Mak. Appl. Manag. Eng. 5, 1–29. https://doi.org/10.31181/dmame0305102022t
Citation: https://doi.org/10.5194/gc-2022-16-AC2
-
AC2: 'Reply on RC1', Hossein Hamidifar, 20 Jun 2023
-
RC2: 'Comment on gc-2022-16', Sebastian G. Mutz, 08 Jun 2023
Disclosure of Expertise
I am familiar with many theoretical aspects of decision making and statistics. However, I have not applied the presented methods or related methods in this type of study. Therefore, I cannot comment on all aspects of this study. My comments and suggestions are written as someone who is interested in exploring the use and potential of such methods in an education setting. Please keep these points in mind when reading my comments.
Summary
The manuscript by Ghorbani et el. describes the application of a hybrid decision-making approach to improve the quality of higher education. Data for nine criteria (or qualitative components relevant to learning) were collected for the application of this approach at four different universities in Iran. In my understanding of the method and study, the approach is merited and can potentially help with informed multi-criteria decision making to improve the quality of education at universities. The manuscript is generally well written, but I do have a few bigger concerns about (and suggestions for) this study/manuscript. Nevertheless, I am confident that the authors will be able to address those, and I look forward to the revised manuscript.
General Comments
1. The study should be clearer on its goal and novelty. In my understanding of the manuscript, I see the novelty in the study primarily in the introduction of the fuzzy aspect and hybrid approach for this particular type of problem. The introduction is well written and introduces the backgrounds nicely. However, toward the end of it, it should be clearer on (a) the novelty of the study, and (b) the need/justification for the proposed method (specifically the fuzzy aspect). In other words, what problems is the study (and novel part of the method) addressing that others did not? I do think the authors could make a very good case for a push toward fuzzy approaches, for example. See specific comments for more details.
2. Generally, I advise that parts of the manuscript are rewritten to allow non-specialists in the specific methods applied here to easily follow the manuscript. These changes would include the addition of brief conceptual descriptions of the methods before the study’s specific applications are detailed. See specific comments for more details.
3. Figures and Tables: I think more informative figures would help convey the work presented here. It is difficult to understand what information figure 2 is meant to convey, for example. Furthermore, it can be tricky to get an overview of the results from just the tables. If the authors can think of a way to visually represent the information, I encourage them to do so. The tables that contain the actual values could then serve as a supplement for readers who wish to have a closer look.
4. Some of the conclusions lack evidence. The benefits of this method (over others) has not been clearly demonstrated or argued.
Specific Comments
L21-22: Here, the authors jump right into weight coefficients and transferring these for the hybrid approach. Those working with AHP may immediately know what is meant. However, I suggest rewriting parts of the abstract to account for a lack of familiarity with the methods among interested readers. I have some knowledge of the mathematical/algorithmic sides of decision making, but it wasn’t clear to me what the weight coefficients referred to until reading the manuscript. Not writing the abstract with the assumption of familiarity with AHP & TOPSIS may also help potential users of the method find an easier entry to it.
L23: The term “relative closeness” in this context also needs a brief explanation. It is a performance criterion for what specifically? Maybe the authors can come up with a way to make this more understandable by explaining it is the closeness to an ideal solution (without getting into too much detail about the method).
L59: Do you mean “pairwise comparison of alternatives”?
L61: Remove underscore after (2010).
L71-76: “[...] data is qualitative, the number of data points is insufficient, the data is insufficiently accurate, or the data is derived from unknown sources” – all these just describe sources of uncertainty. The involvement of “people’s emotions” (which would result in subjective/Bayesian type probabilities/uncertainties) is another example. Before listing these examples, I advise the authors to be a little clearer on what fuzzy set theory/fuzzy mathematics is and what its general strengths are, so the readers can better understand the merits of a fuzzy approach. In contrast to boolean/binary classification or attribution, it deals in degrees (or probability) of attribution. The more general argument can be made that as soon as we have any notable uncertainty in the attribution to any element to any class/group/cluster, a fuzzy approach may be merited and ought to be considered at least. Since the listed goal of the paper is model development, I see some of the novelty of this study in (a) its specific application, and (b) introducing the fuzzy aspects/pushing for the more frequent use of a fuzzy approach to MCDM. Therefore, that aspect should be expanded on a little.
L89: Very minor point: I think the “fame” of the universities (or the status of simply being “well known”, as described later in the text) is less relevant than their good reputation that is a result of their high rankings. I recommend highlighting that (as done in the abstract) instead of their fame.
L95: Klein’s model should be introduced conceptually somewhere. In the introduction, the authors touch on it, but focus more on where it is employed rather than what the model is. I advise the authors to include this somewhere, since they refer to it a lot.
L98: “These elements were researched” is rather vague. What was researched about the elements? I suggest being more specific here to correctly set up the readers expectation of the manuscript.
L101: Please already state the sample size here (i.e., number of WRE students used in the study).
L102: What did those questionnaires, interviews and surveys actually involve? (The appendix should be referenced here and the general nature of the questions should be described). Furthermore, “to collect data” is rather vague. What type of data/information was gathered? Remind the reader also of the goal behind collecting it.
L105: Before jumping into the specific application of the AHP method, it would be most helpful for readers to get a brief, conceptual explanation of the method. I had come across an application of AHP before, yet I struggled to understand everything involved in steps 1-3. Additionally, a justification for the use of the method should be given.
107: “[...] should perform well when comparing all the criteria in a separate pairwise way, […]” is a good example of step descriptions that will likely confuse (a) readers who have not used or read about the AHP method in a while, and (b) readers who are completely unfamiliar with the method and read the manuscript to learn about new criteria ranking and decision making methods for higher education (see L105 comment).
L114-115: (a) “Fuzzy expansion relationships” and (b) where the “weight matrix” fits into the method need to be explained (see L105 comment). Generally, I advise the authors to explain how exactly fuzzy concepts were introduced to the method.
L119-140: I see a similar problem here as in the AHP section. The authors explain more of TOPSIS than AHP, but the problem of lacking explanations persists. TOPSIS needs to be conceptually explained and its use in this study should be justified. What are its weaknesses and strengths (e.g., it is relatively easy to understand)? Why do these method characteristics make it suitable for this particular problem?
L119: I think this sentence can be quite confusing to many. (It only made sense to me, since I work a lot with different types of abstract distances). It’s an example of how the lack of a description (see previous comment) may confuse readers. What type of distance is it? How is the distance between the ideal solution and alternatives being calculated? Before even getting into this, I also advise the authors to explain that an ideal solution is defined (that is not considered fully achievable), and that the goal is to find the closest possible “match”.
L145: I think a figure to help with the explanations above will be extremely useful. However, I do not really understand this particular figure (2). It’s unclear what it is meant to convey. I suggest the authors revise it completely and supplement it with a more informative caption.
L157: Please list the nine qualitative educational characteristics again here, in the table caption and figures 3-4. Since the numbers are referenced a lot, it would be good to have a reference table that the reader can use to look these up while going through the results section.
L233-236: About the Fourier series functions: (a) How does it allow you to determine significance and what type of significance do the authors refer to? (b) While the explanation of a fourier series is correct, I suggest adding a more intuitive explanation for different types of readers.
Figures 3-4: These figures constitute the key results of the study. Once readers understand the methods and calculated metrics, these are informative and easy to understand. I have 2 suggestions for these figures, however: (a) The legend is tricky to read; I advise the use of larger labels. (b) I do not understand the added value of adding the curves to figures 3 and 4. What is gained from the Fourier curves here?
L279: “[…] may better evaluate the quality of education […]”. This statement needs evidence through comparisons. I can see how the approach presented here can be very useful and relatively easy to implement. However, it is not clear why it is better than alternatives. This should be elaborated on through comparisons, evidence and good general arguments.
L289-291: I would welcome the introduction and greater establishment of such methods in other fields, universities and countries. I would therefore be happy to see the suggested changes implemented. They may may help with that exactly.
Citation: https://doi.org/10.5194/gc-2022-16-RC2 -
AC3: 'Reply on RC2', Hossein Hamidifar, 20 Jun 2023
Reviewer #2
Dear Reviewer,
Thank you for your comment. We appreciate your feedback and have made revisions to address your concerns.
Here are the point-by-point responses:
R2C1: The study should be clearer on its goal and novelty. In my understanding of the manuscript, I see the novelty in the study primarily in the introduction of the fuzzy aspect and hybrid approach for this particular type of problem. The introduction is well written and introduces the backgrounds nicely. However, toward the end of it, it should be clearer on (a) the novelty of the study, and (b) the need/justification for the proposed method (specifically the fuzzy aspect). In other words, what problems is the study (and novel part of the method) addressing that others did not? I do think the authors could make a very good case for a push toward fuzzy approaches, for example. See specific comments for more details.
Response: Based on your feedback, we have revised the introduction to provide a clearer explanation of the novelty of the study and the justification for the proposed method. We have emphasized the novelty of introducing the fuzzy aspect and hybrid approach for addressing the specific problem at hand. Additionally, we have highlighted the need for fuzzy approaches in multi-criteria decision-making and the potential benefits they offer compared to traditional methods. This clarification aims to demonstrate the unique contribution of the study and the specific problems it addresses that others have not sufficiently tackled. According to the reviewer’s comment, the following paragraphs have been added to the text:
“Based on the existing evidence (Niu et al., 2019; Muhammad et al., 2020), the growing utilization of the AHP method in educational settings has resulted in the prioritization of more appropriate parameters to improve the quantitative and qualitative aspects of educational institutions and research centres. While alternative methods like FUCOM, BWM, DIBR, or LBWA are also employed by researchers, their application is often limited to specific objectives and constraints. However, in this study, the AHP method was preferred due to its widespread popularity among researchers (Ayough et al., 2023; Chen and Luo, 2023; Jagtap and Karande, 2023; Naz et al., 2023; Sivaprakasam and Angamuthu, 2023).
There are various computational methods available for multi-criteria decision-making, each with its own set of advantages and disadvantages. For instance, the VIKOR method is employed in decision-making processes that utilize compromise programming. This method proves useful when the decision maker struggles to determine the superiority of criteria, whether they are proportional or non-proportional. In such cases, the VIKOR method facilitates the identification of a solution that considers multiple criteria (Paradowski and Sałabun, 2021).
Another method, MABAC, has been proposed specifically for ranking research alternatives. It determines rankings based on the distance from the geometric mean of the available options. However, it is important to note that the applicability of this method is limited to specific scenarios, and the resulting rankings may not hold sufficient value for a given problem (Do and Nguyen, 2022).
MAIRCA is yet another multi-criteria decision-making technique that yields rankings for different options upon completion of computations. Inputs for this method include the decision matrix, criterion weights, and types of criteria. Various concepts such as gap, actual weight, and theoretical weight are incorporated into this method and influence the final ranking. Notably, the best option in this technique is determined by the one with the smallest gap. It can be said that this method follows an extensive process to arrive at a solution (Do and Nguyen, 2022).
Among the newer ranking methods for multi-criteria decision-making problems is the MARCOS method. Similar to the TOPSIS method, it focuses on ranking alternatives by constructing a decision matrix. However, this method alone cannot calculate criterion weights and is typically used as a supplementary approach alongside other techniques like AHP (Duc Trung, 2022). Nevertheless, the TOPSIS method was utilized in this particular study. By employing the TOPSIS method, it becomes possible to identify the best possible answers within the range of problem criteria while duly considering the significance of each criterion (Broniewicz and Ogrodnik, 2021; Tutak and Brodny, 2022). The advantages of this method include its ability to handle both positive and negative criteria, accommodate quantitative and qualitative criteria, and convert qualitative criteria into quantitative measures. Additionally, the computational ease associated with this method is another notable feature (Ayan and Abacioglu, 2022; Do and Nguyen, 2022; İnce and Hakan Isik, 2017; Öztaş et al., 2023).”
“This study aims to demonstrate the application of hybrid multi-criteria decision-making techniques for evaluating the quality of education in higher education institutions. Through sensitivity analysis and charts, it seeks to understand the factors influencing student education and address issues related to educational quality. This provides valuable assistance to researchers and decision-makers, leading to the development of more suitable models for educational development and implementation planning. The study utilizes the Fuzzy-AHP-TOPSIS approach as a powerful MCDM tool for evaluating initial data. This approach improves solution certainty in a fuzzy environment by calculating criteria weights and alternative rankings. The qualitative components of education and learning are analyzed using the Klein model. The proposed methodology offers a comprehensive approach to evaluating teaching and learning quality in specialized fields like WRE. It incorporates diverse criteria, flexibility, high evaluation capability, and appropriate comprehensiveness. The model was implemented in four renowned Iranian universities, evaluating results for eighteen specific cases by varying the number and type of quality criteria. The Fourier series expansion was employed to analyze and investigate the results. Such analyses greatly contribute to better planning and improving the quality of education.”
R2C2: Generally, I advise that parts of the manuscript are rewritten to allow non-specialists in the specific methods applied here to easily follow the manuscript. These changes would include the addition of brief conceptual descriptions of the methods before the study’s specific applications are detailed. See specific comments for more details.
Response: We have considered your suggestion and have added brief conceptual descriptions of the methods employed in the study. These descriptions are provided before the detailed applications of the methods, allowing non-specialists to understand the fundamental concepts and follow the manuscript more easily. By including these explanations, we aim to make the study more accessible to readers who may not be familiar with the specific methods utilized.
R2C3: Figures and Tables: I think more informative figures would help convey the work presented here. It is difficult to understand what information figure 2 is meant to convey, for example. Furthermore, it can be tricky to get an overview of the results from just the tables. If the authors can think of a way to visually represent the information, I encourage them to do so. The tables that contain the actual values could then serve as a supplement for readers who wish to have a closer look.
Response: Thank you for your valuable input regarding the presentation of our work. We have taken your suggestion into consideration and made revisions to Figure 2 to ensure it conveys the information more clearly. While we understand the desire for visual representations, we believe that presenting the results in tables offers several advantages over figures. Firstly, tables provide a concise and organized format for presenting detailed numerical data. They allow for easy comparison and reference, as readers can quickly locate specific values or compare different variables. This structured presentation enhances the clarity and accuracy of the information being conveyed. Additionally, tables provide a comprehensive overview of the results, allowing readers to grasp the complete set of findings at a glance. In contrast, figures may be limited in their capacity to present all relevant details, potentially leading to oversimplification or loss of critical information.
R2C4: Some of the conclusions lack evidence. The benefits of this method (over others) has not been clearly demonstrated or argued.
Response: We acknowledge your comment regarding the need for more evidence to support the conclusions. To address this concern, we have expanded the discussion in the conclusion section, providing more substantial evidence and arguments. We have specifically emphasized the benefits of the proposed method over other approaches, highlighting its ability to handle both positive and negative criteria, accommodate quantitative and qualitative measures, and convert qualitative criteria into quantitative evaluations. These revisions aim to strengthen the conclusions and provide a clearer demonstration of the advantages of the proposed methodology.
R2C5: L21-22: Here, the authors jump right into weight coefficients and transferring these for the hybrid approach. Those working with AHP may immediately know what is meant. However, I suggest rewriting parts of the abstract to account for a lack of familiarity with the methods among interested readers. I have some knowledge of the mathematical/algorithmic sides of decision making, but it wasn’t clear to me what the weight coefficients referred to until reading the manuscript. Not writing the abstract with the assumption of familiarity with AHP & TOPSIS may also help potential users of the method find an easier entry to it.
Response: We have carefully reviewed your suggestion and have revised the abstract accordingly. We have added a brief explanation of weight coefficients and their transfer in the fuzzy environment using the AHP method. This clarification aims to ensure that interested readers, including those unfamiliar with AHP and TOPSIS, can have an easier entry into the study and comprehend the key concepts involved. The following sentences have been added to the Abstract section:
“First, the weight coefficients were determined by surveying the students in a fuzzy environment using the AHP method. This step prioritizes the criteria of the problem based on the experts' perspective. Second, these coefficients were then transferred to the TOPSIS environment, where the previously prioritized criteria are utilized to select the ideal solution to the problem.”
R2C6: L23: The term “relative closeness” in this context also needs a brief explanation. It is a performance criterion for what specifically? Maybe the authors can come up with a way to make this more understandable by explaining it is the closeness to an ideal solution (without getting into too much detail about the method).
Response: Thank you for pointing out the need for a brief explanation of the term "relative closeness." We have addressed this concern by adding an explanation in Step 8 of the methodology section, specifying that it represents the closeness to an ideal solution. This addition aims to provide a clearer understanding of the concept without delving into excessive technical details.
R2C7: L59: Do you mean “pairwise comparison of alternatives”?
Response: We appreciate your observation, and we have made the necessary revision. The term text has been revised in the text as suggested.
R2C8: L61: Remove underscore after (2010).
Response: The underscore after (2010) has been removed.
R2C9: L71-76: “[...] data is qualitative, the number of data points is insufficient, the data is insufficiently accurate, or the data is derived from unknown sources” – all these just describe sources of uncertainty. The involvement of “people’s emotions” (which would result in subjective/Bayesian type probabilities/uncertainties) is another example. Before listing these examples, I advise the authors to be a little clearer on what fuzzy set theory/fuzzy mathematics is and what its general strengths are, so the readers can better understand the merits of a fuzzy approach. In contrast to boolean/binary classification or attribution, it deals in degrees (or probability) of attribution. The more general argument can be made that as soon as we have any notable uncertainty in the attribution to any element to any class/group/cluster, a fuzzy approach may be merited and ought to be considered at least. Since the listed goal of the paper is model development, I see some of the novelty of this study in (a) its specific application, and (b) introducing the fuzzy aspects/pushing for the more frequent use of a fuzzy approach to MCDM. Therefore, that aspect should be expanded on a little.
Response: We have taken your advice into consideration and have revised the explanation of sources of uncertainty. We have provided a more comprehensive explanation of the fuzzy set theory and its general strengths, emphasizing its ability to handle degrees of uncertainty and attribute elements to classes or clusters based on probabilities or degrees of membership. Additionally, we have expanded on the novelty of the study in terms of its specific application and the introduction of fuzzy aspects in multi-criteria decision-making. These revisions aim to provide a more thorough understanding of the merits of the fuzzy approach and the unique contributions of the study.
R2C10: L89: Very minor point: I think the “fame” of the universities (or the status of simply being “well known”, as described later in the text) is less relevant than their good reputation that is a result of their high rankings. I recommend highlighting that (as done in the abstract) instead of their fame.
Response: We agree with your point and have revised the text accordingly. The emphasis has been shifted from the fame of the universities to their good reputation resulting from high rankings. The revised text now highlights the universities' reputation as a reflection of their quality and ranking rather than their mere fame. the text has been revised as below:
“The model was implemented in four out of the top ten universities in Iran, evaluating results for eighteen specific cases by varying the number and type of quality criteria.”
R2C11: L95: Klein’s model should be introduced conceptually somewhere. In the introduction, the authors touch on it, but focus more on where it is employed rather than what the model is. I advise the authors to include this somewhere, since they refer to it a lot.
Response: Thank you for your comment. We understand the importance of introducing Klein's model conceptually to provide a better understanding of its relevance to our study. As our focus is on utilizing the findings of Klein's research and maintaining brevity in writing, we acknowledge that more information about the model would be beneficial. Because of the limitations regarding the number of manuscript pages, we have provided references for readers who wish to explore the model further.
R2C12: L98: “These elements were researched” is rather vague. What was researched about the elements? I suggest being more specific here to correctly set up the readers expectation of the manuscript.
Response: We appreciate your suggestion to be more specific about what was researched regarding the elements. In response, we have revised the text as follows: "All the qualitative criteria proposed by Klein were examined…"
R2C13: L101: Please already state the sample size here (i.e., number of WRE students used in the study).
Response: Thank you for pointing out the need to state the sample size explicitly. We have revised the text as follows: "This study's population consists of 112 WRE students from a diverse range of universities."
R2C14: L102: What did those questionnaires, interviews and surveys actually involve? (The appendix should be referenced here and the general nature of the questions should be described). Furthermore, “to collect data” is rather vague. What type of data/information was gathered? Remind the reader also of the goal behind collecting it.
Response: We acknowledge your comment and agree that providing more information about the questionnaires, interviews, and surveys would enhance the reader's understanding. We have included a reference to the appendix where the questionnaire is provided and have described the nature of the questions in the text. Additionally, we have clarified the goal behind collecting the data, emphasizing that it aims to determine the relative importance of the desired criteria based on participants' specific university conditions. The revised text now reads as follows: "Considering the nature of the AHP method, which is based on pairwise comparisons, the prioritization of alternatives (i.e., the four mentioned universities) and criteria (i.e., the nine qualitative criteria introduced by Klein) has been provided to participants in the form of questionnaires. The participants are asked to complete the questionnaires based on the specific conditions of their own university to determine the relative importance of the desired criteria. The questionnaire (Appendix 1) includes items such as the content of course materials, learning activities, the role of instructors in learning, educational materials and resources, groupings and collective participation, suitable learning environment and time, and assessment."
R2C15 and R2C16: L105: Before jumping into the specific application of the AHP method, it would be most helpful for readers to get a brief, conceptual explanation of the method. I had come across an application of AHP before, yet I struggled to understand everything involved in steps 1-3. Additionally, a justification for the use of the method should be given.
and 107: “[...] should perform well when comparing all the criteria in a separate pairwise way, […]” is a good example of step descriptions that will likely confuse (a) readers who have not used or read about the AHP method in a while, and (b) readers who are completely unfamiliar with the method and read the manuscript to learn about new criteria ranking and decision making methods for higher education (see L105 comment).
Response: We appreciate your feedback. We have provided a justification for using the method, emphasizing its widespread acceptance and effectiveness in ranking criteria and making decisions in various domains. These revisions aim to provide readers, including those unfamiliar with AHP, with a better understanding of the method and its relevance to the study.
R2C17: L114-115: (a) “Fuzzy expansion relationships” and (b) where the “weight matrix” fits into the method need to be explained (see L105 comment). Generally, I advise the authors to explain how exactly fuzzy concepts were introduced to the method.
Response: Thank you for your comment regarding the explanation of fuzzy concepts in our study. We acknowledge the importance of providing clarity on how fuzzy concepts were introduced to the AHP method. While various methods have been proposed by researchers for converting real data into fuzzy data, we utilized simple techniques that have been previously presented in studies by Sirisawat and Kiatcharoenpol (2018) and Zavadskas et al. (2020). We have included references to these studies in the text to allow interested readers to explore the details further. We understand that due to the length constraints of the article, it is not possible to present all the specific details in the text. However, we believe that these references will provide readers with valuable information on the implementation of fuzzy concepts in the AHP method.
R2C18: L119-140: I see a similar problem here as in the AHP section. The authors explain more of TOPSIS than AHP, but the problem of lacking explanations persists. TOPSIS needs to be conceptually explained and its use in this study should be justified. What are its weaknesses and strengths (e.g., it is relatively easy to understand)? Why do these method characteristics make it suitable for this particular problem?
Response: Thank you for your valuable feedback regarding the explanation of the TOPSIS method in our study. We recognize the need for a more comprehensive conceptual explanation of TOPSIS and a justification for its use in this research. In response to your comment and taking into account the length constraints of the manuscript, we have expanded the text to provide more information about the different Multi-Criteria Decision Making (MCDM) methods, including TOPSIS. The following paragraphs have been added to the text:
“Based on the existing evidence (Niu et al., 2019; Muhammad et al., 2020), the growing utilization of the AHP method in educational settings has resulted in the prioritization of more appropriate parameters to improve the quantitative and qualitative aspects of educational institutions and research centres. While alternative methods like FUCOM, BWM, DIBR, or LBWA are also employed by researchers, their application is often limited to specific objectives and constraints. However, in this study, the AHP method was preferred due to its widespread popularity among researchers (Ayough et al., 2023; Chen and Luo, 2023; Jagtap and Karande, 2023; Naz et al., 2023; Sivaprakasam and Angamuthu, 2023).
There are various computational methods available for multi-criteria decision-making, each with its own set of advantages and disadvantages. For instance, the VIKOR method is employed in decision-making processes that utilize compromise programming. This method proves useful when the decision maker struggles to determine the superiority of criteria, whether they are proportional or non-proportional. In such cases, the VIKOR method facilitates the identification of a solution that considers multiple criteria (Paradowski and Sałabun, 2021).
Another method, MABAC, has been proposed specifically for ranking research alternatives. It determines rankings based on the distance from the geometric mean of the available options. However, it is important to note that the applicability of this method is limited to specific scenarios, and the resulting rankings may not hold sufficient value for a given problem (Do and Nguyen, 2022).
MAIRCA is yet another multi-criteria decision-making technique that yields rankings for different options upon completion of computations. Inputs for this method include the decision matrix, criterion weights, and types of criteria. Various concepts such as gap, actual weight, and theoretical weight are incorporated into this method and influence the final ranking. Notably, the best option in this technique is determined by the one with the smallest gap. It can be said that this method follows an extensive process to arrive at a solution (Do and Nguyen, 2022).
Among the newer ranking methods for multi-criteria decision-making problems is the MARCOS method. Similar to the TOPSIS method, it focuses on ranking alternatives by constructing a decision matrix. However, this method alone cannot calculate criterion weights and is typically used as a supplementary approach alongside other techniques like AHP (Duc Trung, 2022). Nevertheless, the TOPSIS method was utilized in this particular study. By employing the TOPSIS method, it becomes possible to identify the best possible answers within the range of problem criteria while duly considering the significance of each criterion (Broniewicz and Ogrodnik, 2021; Tutak and Brodny, 2022). The advantages of this method include its ability to handle both positive and negative criteria, accommodate quantitative and qualitative criteria, and convert qualitative criteria into quantitative measures. Additionally, the computational ease associated with this method is another notable feature (Ayan and Abacioglu, 2022; Do and Nguyen, 2022; İnce and Hakan Isik, 2017; Öztaş et al., 2023).”
R2C19: L119: I think this sentence can be quite confusing to many. (It only made sense to me, since I work a lot with different types of abstract distances). It’s an example of how the lack of a description (see previous comment) may confuse readers. What type of distance is it? How is the distance between the ideal solution and alternatives being calculated? Before even getting into this, I also advise the authors to explain that an ideal solution is defined (that is not considered fully achievable), and that the goal is to find the closest possible “match”.
Response: Thank you for your feedback on the clarity of the sentence regarding the relative closeness coefficient. We understand that the lack of a description may confuse readers who are not familiar with this concept. To address this concern and provide a better understanding, we have revised the sentence as follows:
“The relative closeness coefficient ( ), that represents the closeness to an ideal solution or …”
R2C20: L145: I think a figure to help with the explanations above will be extremely useful. However, I do not really understand this particular figure (2). It’s unclear what it is meant to convey. I suggest the authors revise it completely and supplement it with a more informative caption.
Response: Thank you for your feedback regarding Figure 2. We agree that a visual aid can greatly enhance the understanding of the concepts explained in the manuscript. Based on your suggestion, we have completely revised Figure 2 to make it more informative and helpful. The new figure provides a clear representation of the concepts discussed in the text.
R2C21: L157: Please list the nine qualitative educational characteristics again here, in the table caption and figures 3-4. Since the numbers are referenced a lot, it would be good to have a reference table that the reader can use to look these up while going through the results section.
Response: Thank you for your feedback regarding the listing of the nine qualitative educational characteristics. We understand your concern about having a reference table for easy access to the criteria while reading the results section. To address this, all of the nine criteria are listed in Figure 2 and also in Table 1. However, they are not repeated in the text for brevity and the criteria are referenced with their unique number in other figures and tables.
R2C22: L233-236: About the Fourier series functions: (a) How does it allow you to determine significance and what type of significance do the authors refer to? (b) While the explanation of a fourier series is correct, I suggest adding a more intuitive explanation for different types of readers.
Response: The Fourier series functions are commonly used to fit data in various problems. In the current research, these Fourier series functions have been employed based on data fitting, and they have been briefly explained in the article to enhance the understanding of the results. For example please see Line 287-290: “This characteristic of MCDM approaches allows for the application of Fourier series functions (Dyke, 2014) to determine the significance of criteria and evaluate the situation more clearly. A Fourier series is a periodic function extension in terms of an infinite algebraic sum of sines and cosines functions that form a link between these two types of trigonometric functions.”
R2C23: Figures 3-4: These figures constitute the key results of the study. Once readers understand the methods and calculated metrics, these are informative and easy to understand. I have 2 suggestions for these figures, however: (a) The legend is tricky to read; I advise the use of larger labels. (b) I do not understand the added value of adding the curves to figures 3 and 4. What is gained from the Fourier curves here?
Response: The figures have been updated with new legends for improved readability. As for the curves, they have been included to reduce the uncertainties associated with the data points and enhance the precision in identifying the highest and lowest values.
R2C24: L279: “[…] may better evaluate the quality of education […]”. This statement needs evidence through comparisons. I can see how the approach presented here can be very useful and relatively easy to implement. However, it is not clear why it is better than alternatives. This should be elaborated on through comparisons, evidence and good general arguments.
Response: the mentioned sentence has been deleted for the sake of clarity.
R2C25: L289-291: I would welcome the introduction and greater establishment of such methods in other fields, universities and countries. I would therefore be happy to see the suggested changes implemented. They may help with that exactly.
Response: Thank you for your positive feedback and support for the broader adoption of the proposed methods. We share your enthusiasm for introducing these methods in other fields, universities, and countries. By implementing the suggested changes and further developing these models, we aim to contribute to the advancement and wider utilization of this evaluation framework.
References
Ayan, B., Abacioglu, S., 2022. Bibliometric Analysis of the MCDM Methods in the Last Decade: WASPAS, MABAC, EDAS, CODAS, COCOSO, and MARCOS. Int. J. Bus. Econ. Stud. 4, 65–85. https://doi.org/10.54821/uiecd.1183443
Ayough, A., Shargh, S.B., Khorshidvand, B., 2023. A new integrated approach based on base-criterion and utility additive methods and its application to supplier selection problem. Expert Syst. Appl. 221, 119740. https://doi.org/10.1016/J.ESWA.2023.119740
Broniewicz, E., Ogrodnik, K., 2021. A Comparative Evaluation of Multi-Criteria Analysis Methods for Sustainable Transport. Energies 14, 5100. https://doi.org/10.3390/en14165100
Chen, Z., Luo, W., 2023. An integrated interval type-2 fuzzy rough technique for emergency decision making. Appl. Soft Comput. 137, 110150. https://doi.org/10.1016/J.ASOC.2023.110150
Do, D.T., Nguyen, N.-T., 2022. Applying Cocoso, Mabac, Mairca, Eamr, Topsis and Weight Determination Methods for Multi-Criteria Decision Making in Hole Turning Process. Strojnícky časopis - J. Mech. Eng. 72, 15–40. https://doi.org/10.2478/scjme-2022-0014
Duc Trung, D., 2022. Multi-criteria decision making under the MARCOS method and the weighting methods: applied to milling, grinding and turning processes. Manuf. Rev. 9, 3. https://doi.org/10.1051/mfreview/2022003
Dyke, P., 2014. An Introduction to Laplace Transforms and Fourier Series, in: Springer Undergraduate Mathematics Series, Springer Undergraduate Mathematics Series. Springer London, London. https://doi.org/10.1007/978-1-4471-6395-4
İnce, M., Hakan Isik, A., 2017. AHP-TOPSIS Method for Learning Object Metadata Evaluation Travelling Salesman Problem View project Development Of Integrated Quality Control System For Production Defect Detection By Artificial Vision In Industrial Ceramic Tile Production View project. Int. J. Inf. Educ. Technol. 7, 884–887. https://doi.org/10.18178/ijiet.2017.7.12.989
Jagtap, M., Karande, P., 2023. The m-polar fuzzy set ELECTRE-I with revised Simos’ and AHP weight calculation methods for selection of non-traditional machining processes. Decis. Mak. Appl. Manag. Eng. 6, 240–281. https://doi.org/10.31181/DMAME060129022023J
Muhammad, A., Shaikh, A., Naveed, Q.N., Qureshi, M.R.N., 2020. Factors affecting academic Integrity in E-Learning of Saudi arabian Universities. an Investigation Using Delphi and AHP. IEEE Access 8, 16259–16268. https://doi.org/10.1109/ACCESS.2020.2967499
Naz, S., Akram, M., Hassan, M.M. ul, Fatima, A., 2023. A hybrid DEMATEL-TOPSIS approach using 2-tuple linguistic q -rung orthopair fuzzy information and its application in renewable energy resource selection . Int. J. Inf. Technol. Decis. Mak. https://doi.org/10.1142/S0219622023500323
Niu, B., Liu, Q., Chen, Y., 2019. Research on the university innovation and entrepreneurship education comprehensive evaluation based on AHP method. Int. J. Inf. Educ. Technol. 9, 623–628. https://doi.org/10.18178/IJIET.2019.9.9.1278
Öztaş, T., Aytaç Adalı, E., Tuş, A., Öztaş, G.Z., 2023. Ranking Green Universities from MCDM Perspective: MABAC with Gini Coefficient-based Weighting Method. Process Integr. Optim. Sustain. 7, 163–175. https://doi.org/10.1007/s41660-022-00281-z
Paradowski, B., Sałabun, W., 2021. Are the results of MCDA methods reliable? Selection of materials for thermal energy storage. Procedia Comput. Sci. 192, 1313–1322. https://doi.org/10.1016/j.procs.2021.08.135
Sivaprakasam, P., Angamuthu, M., 2023. Generalized Z-fuzzy soft β-covering based rough matrices and its application to MAGDM problem based on AHP method. Decis. Mak. Appl. Manag. Eng. 6, 134–152. https://doi.org/10.31181/dmame04012023p
Tutak, M., Brodny, J., 2022. Evaluating differences in the Level of Working Conditions between the European Union Member States using TOPSIS method. Decis. Mak. Appl. Manag. Eng. 5, 1–29. https://doi.org/10.31181/dmame0305102022t
Citation: https://doi.org/10.5194/gc-2022-16-AC3
-
AC3: 'Reply on RC2', Hossein Hamidifar, 20 Jun 2023
Viewed
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 478 | 148 | 27 | 653 | 58 | 10 | 9 |
- HTML: 478
- PDF: 148
- XML: 27
- Total: 653
- Supplement: 58
- BibTeX: 10
- EndNote: 9
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
Mohammad Kazem Ghorbani
Nasser Talebbeydokhti
Hossein Hamidifar
Mehrshad Samadi
Michael Nones
Fatemeh Rezaeitavabe
Shabnam Heidarifar
This preprint has been withdrawn.
- Preprint
(915 KB) - Metadata XML
-
Supplement
(390 KB) - BibTeX
- EndNote