Computing a journal meta-ranking using paired comparisons and adaptive lasso estimators

详细信息    查看全文

• 作者：Laura Vana ; Ronald Hochreiter ; Kurt Hornik
• 关键词：Adaptive lasso estimators ; Journal lists ; Meta ; ranking ; Operations research
• 刊名：Scientometrics
• 出版年：2016
• 出版时间：January 2016
• 年：2016
• 卷：106
• 期：1
• 页码：229-251
• 全文大小：537 KB
• 参考文献：Adler, N. J., & Harzing, A. W. (2009). When knowledge wins: Transcending the sense and nonsense of academic rankings. Academy of Management Learning and Education, 8(1), 72–95.CrossRef
  Agresti, A. (2010). Analysis of ordinal categorical data (2nd ed.). Hoboken: Wiley.CrossRef MATH
  Barman, S., Tersine, R. J., & Buckley, M. R. (1991). An empirical assessment of the perceived relevance and quality of POM-related journals by academicians. Journal of Operations Management, 10(2), 194–212.CrossRef
  Barman, S., Hanna, M. D., & LaForge, R. L. (2001). Perceived relevance and quality of POM journals: A decade later. Journal of Operations Management, 19(3), 367–385.CrossRef
  Bergstrom, C., Jevin, D. W., & Wiseman, M. A. (2008). The Eigenfactor (TM) metrics. The Journal of Neuroscience, 28(45), 11,433–11,434.CrossRef
  Bradley, R. A., & Terry, M. E. (1952). Rank analysis of incomplete block designs: I. The method of paired comparisons. Biometrika, 39(3/4), 324–345.MathSciNet CrossRef MATH
  Cattelan, M. (2012). Models for paired comparison data: A review with emphasis on dependent data. Statistical Science, 27(3), 412–433.MathSciNet CrossRef
  Cheang, B., Chu, S. K. W., Li, C., & Lim, A. (2014). OR/MS journals evaluation based on a refined PageRank method: An updated and more comprehensive review. Scientometrics, 100(2), 339–361.CrossRef
  Cook, W. D., Raviv, T., & Richardson, A. J. (2010). Aggregating incomplete lists of journal rankings: An application to academic accounting journals. Accounting Perspectives, 9(3), 217–235.CrossRef
  de Borda, J. C. (1781). Mémoire sur les elections au scrutin. Histoire de l’Académie Royale des Sciences.
  Dwork, C., Kumar, R., Naor, M., & Sivakumar, D. (2001). Rank aggregation methods for the web. In Proceedings of the 10th international conference on World Wide Web (pp. 613–622). ACM.
  Emond, E. J., & Mason, D. W. (2002). A new rank correlation coefficient with application to the consensus ranking problem. Journal of Multi-Criteria Decision Analysis, 11(1), 17–28.CrossRef MATH
  Fan, J., & Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96(456), 1348–1360.MathSciNet CrossRef MATH
  Firth, D. (2015) qvcalc: Quasi variances for factor effects in statistical models. http://​CRAN.​R-project.​org/​package=​qvcalc . R package version 0.8-9.
  Firth, D., & De Menezes, R. X. (2004). Quasi-variances. Biometrika, 91(1), 65–80.MathSciNet CrossRef MATH
  Frey, B. S. (2010). Withering academia? CESifo working paper series no 3209. SSRN. http://​ssrn.​com/​abstract=​1694448 .
  Frey, B. S., & Rost, K. (2010). Do rankings reflect research quality? Journal of Applied Economics, 13(1), 1–38.CrossRef
  Fry, T. D., & Donohue, J. M. (2013). Outlets for operations management research: A DEA assessment of journal quality and rankings. International Journal of Production Research, 51(23–24), 7501–7526.CrossRef
  Fry, T. D., & Donohue, J. M. (2014). Exploring the author affiliation index. Scientometrics, 98(3), 1647–1667.CrossRef
  Garfield, E. (1972). Citation analysis as a tool in journal evaluation. Science, 178, 471–479.CrossRef
  Geary, J., Marriott, L., & Rowlinson, M. (2004). Journal rankings in business and management and the 2001 research assessment exercise in the UK. British Journal of Management, 15(2), 95–141.CrossRef
  Glänzel, W., & Moed, H. F. (2002). Journal impact measures in bibliometric research. Scientometrics, 53(2), 171–193.CrossRef
  Glänzel, W., & Moed, H. F. (2013). Opinion paper: Thoughts and facts on bibliometric indicators. Scientometrics, 96(1), 381–394.CrossRef
  Goh, C., Holsapple, C., Johnson, L., & Tanner, J. (1996). An empirical assessment of influences on POM research. Omega, 24(3), 337–345.CrossRef
  González-Pereira, B., Guerrero-Bote, V. P., & Moya-Anegón, F. (2010). A new approach to the metric of journals scientific prestige: The SJR indicator. Journal of informetrics, 4(3), 379–391.CrossRef
  Halkos, G. E., & Tzeremes, N. G. (2011). Measuring economic journals citation efficiency: A data envelopment analysis approach. Scientometrics, 88(3), 979–1001.CrossRef
  Harzing, A. W. (2014). Journal quality list (52nd ed.). http://​www.​harzing.​com .
  Holsapple, C. W., & Lee-Post, A. (2010). Behavior-based analysis of knowledge dissemination channels in operations management. Omega, 38(3), 167–178.CrossRef
  Hornik, K., Jankowitsch, R., Lingo, M., Pichler, S., & Winkler, G. (2007). Validation of credit rating systems using multi-rater information. Journal of Credit Risk, 3(4), 1–27.
  Kemeny, J. G., & Snell, L. (1962). Mathematical models in the social sciences. Cambridge: MIT Press.MATH
  List, C. (2013). Social choice theory. In E. N. Zalta (Ed.), The Stanford Encyclopedia of Philosophy, (Winter 2013 ed.). Stanford: The Metaphysics Research Lab.
  Lozano, S., & Salmerón, J. L. (2005). Data envelopment analysis of OR/MS journals. Scientometrics, 64(2), 133–150.CrossRef
  Masarotto, G., & Varin, C. (2012). The ranking lasso and its application to sport tournaments. The Annals of Applied Statistics, 6(4), 1949–1970.MathSciNet CrossRef MATH
  Meredith, J. R., Steward, M. D., & Lewis, B. R. (2011). Knowledge dissemination in operations management: Published perceptions versus academic reality. Omega, 39(4), 435–446.CrossRef
  Mingers, J., & Harzing, A. W. (2007). Ranking journals in business and management: A statistical analysis of the Harzing data set. European Journal of Information Systems, 16(4), 303–316.CrossRef
  Moed, F. (2011). The source normalized impact per paper is a valid and sophisticated indicator of journal citation impact. JASIST, 62, 211–213.CrossRef
  Olson, J. E. (2005). Top-25-business-school professors rate journals in operations management and related fields. Interfaces, 35(4), 323–338.MathSciNet CrossRef
  Page, L., Brin, S., Motwani, R., & Winograd, T. (1999). The pageRank citation ranking: Bringing order to the web technical report. Stanford Digital Libraries Project.
  Petersen, C. G., Aase, G. R., & Heiser, D. R. (2011). Journal ranking analyses of operations management research. International Journal of Operations and Production Management, 31(4), 405–422.CrossRef
  Pinski, G., & Narin, F. (1976). Citation influence for journal aggregates of scientific publications: Theory, with application to the literature of physics. Information Processing and Management, 12(5), 297–312.CrossRef
  R Core Team. (2015). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. http://​www.​R-project.​org/​ .
  Serenko, A., & Bontis, N. (2013). Global ranking of knowledge management and intellectual capital academic journals: 2013 update. Journal of Knowledge Management, 17(2), 307–326.CrossRef
  Serenko, A., & Dohan, M. (2011). Comparing the expert survey and citation impact journal ranking methods: Example from the field of Artificial Intelligence. Journal of Informetrics, 5(4), 629–648.CrossRef
  Theußl, S., Reutterer, T., & Hornik, K. (2014). How to derive consensus among various marketing journal rankings? Journal of Business Research, 67(5), 998–1006.CrossRef
  Thomson Reuters. (2014). The 2014 Journal Citation Reports ® (JCR), Social Sciences Edition. Philadelphia: Thomson Reuters.
  Tibshirani, R. J. (2011). The solution path of the generalized lasso. Redwood City: Stanford University.MATH
  Tüselmann, H., Sinkovics, R. R., & Pishchulov, G. (2015). Towards a consolidation of worldwide journal rankings—A classification using random forests and aggregate rating via data envelopment analysis. Omega, 51, 11–23.CrossRef
  Vokurka, R. (1996). The relative importance of journals used in operations management research: A citation analysis. Journal of Operations Management, 14(4), 345–355.CrossRef
  Willmott, H. (2011). Journal list fetishism and the perversion of scholarship: Reactivity and the ABS list. Organization, 18(4), 429–442.CrossRef
  Xu, F., Liu, W., & Rousseau, R. (2015). Introducing sub-impact factor (SIF-) sequences and an aggregated SIF-indicator for journal ranking. Scientometrics, 102(2), 1577–1593.CrossRef
  Xu, Z., Cheang, B., Lim, A., & Wen, Q. (2011). Evaluating OR/MS journals via PageRank. Interfaces, 41(4), 375–388.CrossRef
  Young, H. P. (1986). Optimal ranking and choice from pairwise comparisons. Information pooling and group decision making, pp. 113–122.
  Zhou, D., Ma, J., & Turban, E. (2001). Journal quality assessment: An integrated subjective and objective approach. IEEE Transactions on Engineering Management, 48(4), 479–490.CrossRef
  Zou, H. (2006). The adaptive lasso and its oracle properties. Journal of the American Statistical Association, 101(476), 1418–1429.MathSciNet CrossRef MATH
  
• 作者单位：Laura Vana (1)
  Ronald Hochreiter (1)
  Kurt Hornik (1)
  
  1. Institute for Statistics and Mathematics, WU Vienna University of Economics and Business, Welthandelsplatz 1, 1020, Vienna, Austria
  
• 刊物主题：Information Storage and Retrieval; Library Science; Interdisciplinary Studies;
• 出版者：Springer Netherlands
• ISSN：1588-2861

文摘

In a “publish-or-perish culture”, the ranking of scientific journals plays a central role in assessing the performance in the current research environment. With a wide range of existing methods for deriving journal rankings, meta-rankings have gained popularity as a means of aggregating different information sources. In this paper, we propose a method to create a meta-ranking using heterogeneous journal rankings. Employing a parametric model for paired comparison data we estimate quality scores for 58 journals in the OR/MS/POM community, which together with a shrinkage procedure allows for the identification of clusters of journals with similar quality. The use of paired comparisons provides a flexible framework for deriving an aggregated score while eliminating the problem of missing data.