From PsychWiki - A Collaborative Psychology Wiki
listen to mcfly music videos microsoft office trial version crack janet jackson halftime show movie remote control cars videos powerpuff girls movie in japan typhoons video singing in the rain movie reviews pirateart mtv video music awards 50 cent fat joe privacy act video surveillance movie down loads map pirate torrent search kjell video plagiarism videos ÃÂ¢ÃÂÃÂº Have you ever wanted to learn about meta-analyses or conduct a meta-analysis but didn't know where to start? This webpage is devoted to providing you expert opinion on what you need to know to start your own meta-analysis.
ÃÂ¢ÃÂÃÂº With the thousands of meta-analyses conducted in all areas of psychology over the past few decades, there has been an ever-increasing number of articles, books, and software programs devoted to how to conduct a meta-analysis. Below, you can find out which of the many sources of information are the most useful and why -- so that you have an easy-to-use starting place for learning everything about meta-analytic reviews.
Where should I start?
Conducting a meta-analysis is easier than it appears. While there are many, many books that describe all the intricacies of conducting a meta-analysis, try not to lose sight of the fact that a meta-analysis is essentially a straightforward process of collecting a group of studies that focus on a shared topic, and then entering statistical information into software designed to conduct meta-analysis (see choose your statistical software). The software tells you the average effect sizes from your group of studies, and also analyzes moderating variables, if that is something you are interested in examining.
If you want to learn what is a meta-analysis...
If you want to learn how to start conducting a meta...
What is a meta-analysis?
Three Basic Questions
- A meta-analysis answers three general questions:
- Central tendency ÃÂ¢ÃÂÃÂ The central purpose of a meta analysis is to test the relationship between two variables such that X affects Y. Central tendency refers to identifying whether X affects Y via statistically summarizing signficance levels, effect sizes, and/or confidence intervals. You are trying to answer whether X affects Y, is the effect significant, and how strong is that effect?
- Variability ÃÂ¢ÃÂÃÂ There is always going to be some degree of variation between the outcomes of the individual studies that compose the meta-analysis. The question is whether the degree of variability is signficantly different than what we would expect by chance alone. If so, then its called heterogeneity. (for more info on heterogeneity, see here and here and here)
- Prediction ÃÂ¢ÃÂÃÂ If there is heterogeneity (variability), then we look for moderating variables that explain the variability. In other words, does the effect of X on Y differ with moderator variables?
Five Basic Steps
- There are generally five separate steps in conducting a meta-analysis:
- Define your hypothesis ÃÂ¢ÃÂÃÂ First you must have a well-defined statement of the relationship between the variables under investigation so that you can carefully define the inclusion and exclusion criteria when locating potential studies. For more information see Chapter 2 of (Lipsey & Wilson, 2001) (Practical Meta-Anlaysis) for a thorough examination of this step.
- Locate the studies ÃÂ¢ÃÂÃÂ A meta-analysis is only informative if it adequately summarizes the existing literature, so a thorough literature search is critical to retrieve every relevant study, such as database searches, ancestry approach, descendancy approach, hand searching, and the invisible college (i.e., network of researchers who know about unpublished studies, conference proceedings, etc). For more information see (Johnson & Eagly, 2000) (Handbook of Research Methods in Social and Personality Psychology) which details five general ways to retrieve relevant articles.
- Input data ÃÂ¢ÃÂÃÂ Gather empiricial findings from primary studies (e.g., p-value, effect size, etc) and input into statistical database. Not every study provides sufficient statistical information to calculate the effect size statistic. For more information see below about choosing your statistical software.
- Cacluate effect sizes ÃÂ¢ÃÂÃÂ Calculate the overall effect by converting all statistics to a common metric, making adjustments as necessary to correct for issues like sample-size or bias, and then calculating central tendency (e.g., mean effect size and confidence intervals around that effect size) and variability (e.g., heterogeneity analysis). For more information see below about choosing which effect size index to calculate and see (Lipsey & Wilson, 2001) (Practical Meta-Anlaysis) for all the different statistical formulas.
- Analyze variables ÃÂ¢ÃÂÃÂ If heterogeneity exists, you may want to analyze moderating variables by coding each variable in the database and analyzing either mean differences (for categorical variables) or weighted regression (for continuous variables) to see if the variable accounts for the variability in the effect size. Note - even if heterogeneity does not exist, some argue analyzing moderating variables is appropriate ((Rosenthal, 1995)). FYI - (Rosenthal, 1995) is also an excellent Psychological Bulletin article on how to write a meta-analysis.
How do I conduct a meta-analysis?
First, choose which statistical approach suits your needs
- Hedges & Olkin Approach ÃÂ¢ÃÂÃÂ see (Hedges, 1981); (Hedges, 1982); (Hedges & Olkin, 1985)
- Rosenthal & Rubin Approach ÃÂ¢ÃÂÃÂ see (Rosenthal, 1991); (Rosenthal & Rubin, 1978); (Rosenthal & Rubin, 1988)
- Hunter, Schmidt, & Jackson - see (Hunter, Schmidt, & Jackson, 1982); (Hunter & Schmidt, 1990)
Second, choose which effect size index to calculate
- The r family ÃÂ¢ÃÂÃÂ Correlation Coefficient - The "r" family includes all types of correlation coefficients (e.g., r, phi, rho, etc) and (Johnson & Eagly, 2000) suggest using r when the studies composing the meta analysis primarily report the correlation between variables, but also see (Rosenthal & DiMatteo, 2001) for a discussion of the advantages of using r over d.
- The d family ÃÂ¢ÃÂÃÂ Standardized Difference - The "d" family includes Cohen's d (unweighted) and Hedges g (weighted), and (Johnson & Eagly, 2000) suggest using d when the studies composing the meta-analysis primarily report ANOVAs and t-tests comparisons between groups.
(for an online effect size calculator for both "r" and "d", see the Larry C. Lyons website)
Third, choose your statistical software
- You have two basic options -- use specialized software designed to conduct meta-analyses, or use standard statistical software such as SPSS and SAS. There are pros/cons to whichever option you use, so how do you choose? What you need are opinions/suggestions from those who have already used each type of software, which is where PsychWiki comes in.
- (this list is not exhaustive, so add any other software you think is useful)
- SPSS and SAS (free)
- MIX (free)
- MIX - Meta-analysis with Interactive eXplanations - is an Excel-based software that can be found here.
- Meta-Analysis (free)
- META (Meta-Analysis Easy to Answer) (free)
- Meta-Analysis Calculator (free)
- CMA (Comprehensive Meta-Analysis) (free demo, academic pricing)
- Metawin (free demo, student discounts)
- DSTAT (free demo, price $25)
- Advanced Basic Meta-analysis
- Developed by (Mullen, 1989) ...
- Developed by (Stauffer, 1996) ...
- R-Project (free, open source)
- Developed by collaboration, see the website for details. Meta package must be installed and loaded separately.
websites you may find interesting or helpful...
- For an online slide-show of how to conduct a meta-analysis,
- see the University of Pittsburgh's Supercourse on how to conduct a meta-analysis.
- For a powerpoint presentation summary of the (Lipsey and Wilson, 2001) book Practical Meta-analysis,
- see the David B. Wilson website.
- For a concise depiction of the meta-analytic process,
- For a truly engaging and informative paper on the history of meta-analyses written by the person who coined the term "meta-analysis"
- see Gene V. Glass website.
- For a discussion of how a meta-analysis fits into the research process,
- see the CMA (Comprehensive Meta-Analysis) website.
- For a listing of various commercial and freely available meta-analysis software,
- For a 2007 review of meta-analysis software,
- see Bax et al., 2007.
- For a listing of articles that review/compare different meta-analytic software,
- For an online professional development course on how to conduct a meta-analysis,
- see the Statistics.com website.
- For the wikipedia webpage devoted to meta-analysis,
- see this page.
- For a concise summary of the advantages and flaws of a meta-analysis:
- see Medical Communications EBM page.
- For a Fail-Safe Number Calculator (and a paper describing the Fail-Safe Number issue),
- see the Michael S. Rosenberg website.
ÃÂ¢ÃÂÃÂ Back to Research Tools mainpage