A research design is a detailed, proposed plan for a research paper and report. A good design lays out in detail several things: the question, the theoretical framework, previous research, data collection and analysis, interpretation of data and a conclusion bringing everything together. This is a great challenge. The best means of critiquing such a design is to separate each step and see if the presented variables are well-defined and appropriate to the overall project.
Read and understand the basic research question. Then read the theoretical section. This is the basic introduction to the project where the research question is explained in detail. Do not separate the theory from the question being asked; they are closely related. The entire purpose of the theory section is to contextualize the research question. The basic purpose here is to see how the question can be explicated and made clearer. The critique at this stage should address how the theory explains the question.
Read the literature review. Make sure that the articles and books used reflect the theoretical design of the proposed report. Clarify carefully that the papers are recent and reflect state-of-the-art knowledge. Relevance is also important. All literature must relate to the question and theory.
Go over each variable one by one. Start with the dependent variable, that which is being explained, and then analyze each independent variable. Make sure that each variable is well-defined, and that there are well-built “walls” among the variables. If variables overlap each other, then the research paper itself will be flawed. In other words, if you see two variables, one called “social class” and the other “income,” start to worry. This is because these two variables overlap in meaning to a great degree and, as these are really one variable, they can distort the final findings and analysis. Examining the variables is one of the most important parts of a critique.
Analyze the proposed methods of analysis. If the report is statistical, make sure the proper programs are being used to crunch the numbers. Ask about the use of statistical diagnostic programs. These are designed to detect flaws in the analysis method and should be incorporated into any serious report. If factor analysis is being used, make sure these factors are not exceptionally hypothetical, which is a common problem among factor analysis papers. Inferred variables must be closely related to the measured ones.
Look closely at the expected interpretations and conclusions. It is OK for a design to be tentative in these areas, and the phrase “I expect to” will be used a lot. The central issue in the critique here is to make sure the proposed conclusions are close to the data and analysis being proposed. For example, if the paper is about income and preference for the Republican party, it is proper to say that high incomes are correlated with Republican preferences, but improper to conclude that these people must live in the suburbs.