Empirical Study on Applying Meta-analysis Methods in the Social Sciences and Psychological Research

在社会科学和心理学研究中，同一研究问题的多元研究结果不一致，甚至截然相反的现象屡见不鲜。将独立的研究结果整合成一个连贯而全面的整体，涉及到对知识有效性至关重要的推理，就像从原始数据分析中得出结论所涉及的推理一样。作为一种研究整合的方法，元分析被认为是有用的和重要的，在许多国家迅速得到认可并流行起来。然而，在日本，元分析方法还没有被广泛应用于社会科学和心理学研究。元分析的过程可以被概念化为五个阶段：（1）问题的形成，（2）数据收集或文献检索，（3）数据评估：文献编码，（4）分析和解释，（5）结果的呈现。在这项研究中，我们特别关注了 ...更多信息 在（2）数据收集部分，采用PsychInfo、Scopus和CJP（Citation Database for Japanese Papers）等文献计量学方法，从研究论文中提取引文索引、作者索引、引文数量和关键词等，对相关文献进行筛选。本研究对文献综合检索的有效策略进行了探讨。对于第四阶段，我们关注的是效应量，这是荟萃分析中统计方法的基础。在社会科学和心理学研究中，假设显著性检验似乎被过分强调。在我们的研究中，我们已经证明了报告效应量的重要性，这些效应量包括人们永远无法从显著性检验中获得的信息。这些研究的结果在日本学术团体的一些年度会议上发表。各种因素可能会阻碍荟萃分析方法的使用，其中一个原因可能是目前没有软件可以轻松计算效应量和置信区间;正如我们所知，没有计算机几乎不可能计算这些值。在这项研究中，我们开发了一个Excel宏计算机程序，估计周围的标准化均值差6的置信区间的基础上自助BC_a程序。我们的模拟结果表明，我们的程序可以精确地估计置信区间，除非同时发生以下所有情况：正态性假设被严重违反，效果sizp很大，样本量很小。少

In the social sciences and psychological research, it is not unusual for plural research with the same research problems to have inconsistent results, or even have diametrically apposite results. Integrating separate research results into a coherent and comprehensive whole involves inferences as central to the validity of knowledge as the inferences involved in drawing conclusions from primary data analysis. As a method of research integrating, meta-analysis is recognized as useful and important, that it has rapidly gained acceptance and becomes popular in many foreign countries.However in Japan meta-analysis methods are still not being widely used in the social sciences and psychological research The process of meta-analysis can be conceptualized as containing five stages: (1) problem formulation, (2) data collection or the literature search, (3) data evaluation: literature coding, (4) analysis and interpretation, and (5) presentation of results. In this study, we especially focused o … More n stages (2) and (4) which are considered as big obstacles to practice the meta-analysis methods for Japanese researchers.For (2) data collection part, we used PsychInfo, Scopus and CJP (Citation Database for Japanese Papers) for selecting the relevant literatures based on bibliometric strategies, such as by using citation indexing, author indexing, number of citations and keywords extracting from the research papers. The effective strategies for searching the literatures comprehensively were investigated and discussed in our study. Concerning the stage (4), we paid attention to effect sizes, the base of the statistical methods in meta-analysis. In the social sciences and psychological research it seems that hypothesis significance testing has been given too much emphasis. In our study, we have shown the importance of reporting effect sizes which indude those information people can never obtain just from significance testing. The results of the studies were presented in a number of annual meetings ofJapanese academic societies.Various factors may hinder using meta-analysis methods, and one of the reasons could be that there is currently no software available for calculating effect sizes and confidence intervals easily ; as we know it is practically impossible to calculate the values without computers. In this study, we developed an Excel macro computer program to estimate confidence intervals around the standardized mean difference 6 based on the bootstrap BC_a procedure. The results of our simulation show that our program can precisely estimate confidence intervals except when all the following situations happen simultaneously : the normality assumption is extremely violated, effect sizp is large, and sample size is very small. Less