M2R “Development Economics” Research Methodology - Panth´eon Sorbonne University

Research Methodology 2: Writing a good research paper Remi Bazillier [email protected]

Semester 1, Academic year 2017-2018

1 / 71

Introduction

I

What is a good research paper?

2 / 71

Introduction I

This is a paper that... I

... is based on a good research project;

I

... is very clearly written: I

EACH sentence is short, meaningful and is logically linked to the other sentences;

I

A fellow graduate student should be able to reproduce EVERY number or regressions in your papers simply from the information you convey in your paper... this means that you have to be complete and NEVER allusive;

3 / 71

Introduction

I

This is a paper that (continued)... I

... is short: I

As you write your paper, continually ask yourself: “Can I express the same idea in less space?”;

I

Your paper shouldn’t exceed 40 pages (tables and figures included), knowing that you have to abide by the following constraints: 12-point Times New Roman or similar font; 1.5 line spacing; margins of 2.5 cm on the top, bottom, and sides;

4 / 71

Introduction

I

This is a paper that (continued)... I

... is as simple as possible: “Simple is better. Most students think they have to dress up a paper to look impressive. The exact opposite is true. The less math used, the better. The simpler the estimation technique, the better.” (Cochrane, 2005, “Writing tips for PhD students”)

5 / 71

Introduction I

This is a paper that (continued)... I

... is organized in the following way: 1. Title page 2. Introduction 3. Related literature 4. Data or Empirical strategy 5. Empirical strategy or Data 6. Results 7. Robustness checks 8. Conclusion 9. References 6 / 71

Introduction

I

In the next sections, we provide: I

advices and examples for writing each of these 9 sections,

I

as well as additional writing tips.

7 / 71

1. Title page I

A title page is composed of I

a title;

I

your first name and last name, your academic affiliation, and your email;

I

an abstract;

I

keywords;

I

JEL classification codes;

I

acknowledgments.

8 / 71

1. Title page I

The title should present your research question in an eye-catching way: I

“Good bye Lenin (or not?): the effect of communism on people’s preferences”

I

“The slave trade and the origin of mistrust in Africa”

I

“Why do some countries produce so much more output per worker than others?”

I

“Does teacher truancy prevent students from learning? Evidence from Sub-Saharan Africa” (the title of the paper based on Maria Kuecken’s Master 2 thesis)

I

... etc

9 / 71

1. Title page I

The function of the abstract is I

to communicate the main result of your paper in a way that...

I

... will convince the readers that they should proceed with the rest of the paper.

I

So be concise (obey the 100-150 words limit) but appealing.

I

You should write the abstract after the conclusion and the introduction, which are themselves written after the body of the paper (section 3 “Related literature” to section 7 “Robustness checks”).

10 / 71

1. Title page I

The abstract should be followed by: I I

I

The keywords and the JEL classification codes: I I

I

a set of keywords (they shouldn’t exceed 2 lines); a set of JEL classification codes (a system originated by the Journal of Economic Literature). They allow to communicate the economic subfields that are related to your research.

not only convey critical information for the reader but also facilitate the use of your paper by search engines like Econlit (a searchable data base of citations for articles, books, reviews, dissertations, and working papers classified by JEL codes for the years from 1969).

The JEL classification codes are available here: http://www.aeaweb.org/jel/jel_class_system.php

11 / 71

1. Title page I

Abstract of “Does teacher truancy prevent students from learning? Evidence from Sub-Saharan Africa”: Abstract We estimate the impact of teacher truancy on the reading and mathematics achievement of primary school students in eight African countries. Specifically, we study the frequency of teacher tardiness, absenteeism, and skipping class at the school level. An OLS approach reveals that teacher truancy has no average effect on test scores. Rather, it is detrimental only to students from households belonging to the uppermost percentiles of the socioeconomic distribution. An IV approach based on historic exposure of ethnic groups to the slave trade confirms these findings. Our results suggest that it is solely when other poverty-induced constraints have been lifted that the attendance of teachers plays a significant role in determining educational outcomes.

12 / 71

1. Title page

I

Keywords of “Does teacher truancy prevent students from learning? Evidence from Sub-Saharan Africa”:

Keywords: Absenteeism, Educational quality, Slave trade, Trust, Sub-Saharan Africa

13 / 71

1. Title page I

JEL codes for “Does teacher truancy prevent students from learning? Evidence from Sub-Saharan Africa”:

I

Description: I

A12: “General Economics and Teaching” (A)...“General Economics” (A1)... “Relation of Economics to Other Disciplines” (here: educational research) (A12);

I

C36: “Mathematical and Quantitative Methods” (C)... “Multiple or Simultaneous Equation Models; Multiple Variables” (C3)... “Instrumental Variables (IV) Estimation” (C36);

I

I21: “Health, Education, and Welfare” (I)... “Education and Research Institutions” (I2)... “Analysis of Education” (I21). 14 / 71

1. Title page

I

Finally, it is appropriate to thank people who have helped you in the author footnote.

I

For instance: “I thank X, Y, and Z for... I

... their generous guidance and helpful comments.”

I

... their constant support and feedback.”

I

... etc

15 / 71

1. Title page I

Note that you will have to dedicate the second page of your Master 2 thesis to the following foreword: “The University of Paris 1 Panth´eon-Sorbonne neither approves nor disapproves of the opinions expressed in this dissertation: they should be considered as the author’s own.”

I

Traduction: “L’universit´e de Paris 1 Panth´eon-Sorbonne n’entend donner aucune approbation ni d´esapprobation aux opinions ´emises dans ce m´emoire: elles doivent ˆetre consid´er´ees comme propre `a leur auteur.”

16 / 71

2. Introduction

I

The introduction is certainly the most important section of your Master 2 thesis since it constitutes readers’ (and notably referees’) starter.

I

“If an apple does not taste good at the first bite, one simply throws it away without giving any thought on the nutritional value hidden in the apple. Likewise, most referees make up their mind at the first bite, i.e., within 15 minutes of reading a paper” (Kwan Choi, 2002, “How to publish in top journals”).

I

So do an excellent job here!

17 / 71

2. Introduction

I

Your introduction needs to address three sets of questions.

I

First, the WHAT set of questions: 1. what is your research question? (WHAT 1) 2. what is new (i.e. what is your contribution to the literature)? (WHAT 2) 3. what are your findings? (WHAT 3)

18 / 71

2. Introduction I

Second, the WHY set of questions (i.e. WHY is it interesting?): 1. why is the answer consequential? (WHY 1) 2. why isn’t the answer obvious? (WHY 2)

I

Third, the HOW set of questions: 1. how do you proceed regarding the data? (HOW 1) 2. how do you proceed regarding the empirical strategy? (HOW 2)

19 / 71

2. Introduction

I

To achieve this goal here is how you can organize your introduction.

I

Note that you are not requested to strictly abide by the order in which we answer the questions below.

I

However, for your introduction to be good, it must address ALL these questions.

20 / 71

2. Introduction

I

Start with presenting the real world context your research question is related to.

I

This contextualization should rely on real world examples, citations, statistics or anecdotes so that the reader cannot say the paper is disconnected from reality.

21 / 71

2. Introduction

I

“Without this sound footing in the real world, your paper may give the impression to readers that it provides a profound solution to nonexistent problems” (Kwan Choi, 2002, “How to publish in top journals”).

I

It is better if this contextualization does not exceed one paragraph.

22 / 71

2. Introduction

I

Paragraph of contextualization for “Does teacher truancy prevent students from learning? Evidence from Sub-Saharan Africa”: see the 1st paragraph of the introduction

23 / 71

2. Introduction

I

Then answer question WHAT 1 (i.e. present the objective of your paper).

I

Answer to WHAT 1 in “Does teacher truancy prevent students from learning? Evidence from Sub-Saharan Africa”: see the 1st sentence of the 2nd paragraph of the introduction

24 / 71

2. Introduction

I

Then answer question WHY 1 (i.e. explain why the answer to your research question is consequential):

I

Answer to WHY 1 in “Does teacher truancy prevent students from learning? Evidence from Sub-Saharan Africa”: see the 2nd sentence of the 2nd paragraph of the introduction

25 / 71

2. Introduction I

Then answer question WHY 2 (i.e. explain why the answer to your research question is not obvious): 1. if your objective is to estimate the magnitude of the impact of A on B, talk about: I

the endogeneity problems that could explain why this impact may not exist and/or why the magnitude of this impact may be strongly biased in case simple OLS estimation is conducted;

I

the circumstances that could explain why this impact may be heterogeneous across various populations

I

... etc

2. if your objective is to test the sign of the impact of A on B, present the theories hat could explain both a negative and positive impact of A on B... etc 26 / 71

2. Introduction

I

Answer to WHY 2 in “Does teacher truancy prevent students from learning? Evidence from Sub-Saharan Africa”: see the 3rd and 4th paragraph of the introduction

27 / 71

2. Introduction

I

Then answer question WHAT 2 (i.e. explain your contribution to the literature):

I

Answer to WHAT 2 in “Does teacher truancy prevent students from learning? Evidence from Sub-Saharan Africa”: see the 5th paragraph of the introduction

28 / 71

2. Introduction

I

Then answer question HOW 1 (i.e. present your data).

I

Answer to HOW 1 in “Does teacher truancy prevent students from learning? Evidence from Sub-Saharan Africa”: see the 6th paragraph of the introduction

29 / 71

2. Introduction

I

Then answer question HOW 2 (i.e. present your empirical strategy).

I

Be clear about why it allows to solve endogeneity problems.

I

For instance, if you rely on an instrumental variable approach, explain briefly why you think your instrument is valid.

30 / 71

2. Introduction

I

Answer to HOW 2 in “Does teacher truancy prevent students from learning? Evidence from Sub-Saharan Africa”: see the 7th, 8th and 9th paragraph of the introduction

31 / 71

2. Introduction

I

Then answer question WHAT 3 (i.e. present your results).

I

Answer to WHAT 3 in “Does teacher truancy prevent students from learning? Evidence from Sub-Saharan Africa”: the presentation of the results is embedded in the 8th and 9th paragraph of the introduction

32 / 71

2. Introduction I

Do not only focus on the statistical significance of your results (i.e. the fact that their magnitude is large and/or systematic enough to ensure that it was not uncovered by chance).

I

Discuss their magnitude as well and put this magnitude in perspective.

I

You can do this in the introduction, or in another section of your Master 2 thesis you find more adequate.

I

For instance, in “Does teacher truancy prevent students from learning? Evidence from Sub-Saharan Africa”, this discussion can be found in the last paragraph of section 5.3.2.

33 / 71

2. Introduction

I

Last but not least, conclude your introduction with the roadmap of your paper.

I

Roadmap for “Does teacher truancy prevent students from learning? Evidence from Sub-Saharan Africa”: see the last paragraph of the introduction

34 / 71

2. Introduction

I

Five pages is the upper limit for the introduction.

I

Your introduction should be crystal-clear, such that even a non specialist or someone who turns to your topic for the first time, will understand all your points very easily.

I

Note that the preparation of your research proposal that is due on February 3, 2017 will considerably ease the writing of the introduction of your Master 2 thesis.

I

So dedicate time and intellectual energy to this proposal!

35 / 71

2. Introduction

I

Starting from the introduction, your task is now to provide the reader with your main results as fast as possible: “There should be nothing before the main result that a reader does not need to know in order to understand the main result” (Cochrane, 2005, “Writing tips for PhD students”).

I

Never be off-topic!

36 / 71

3. Related literature

I

This section is critical to make the reader better grasp your contribution to the literature.

I

Summarize this literature by decomposing it in no more than 3 strands.

I

Only cite and present the papers that are closely linked to your research question.

37 / 71

3. Related literature I

I

While doing so: I

use your own clear words (do not give the impression that you copied and pasted the abstract of previous papers!!!);

I

explain how your work improve upon these papers, but without making any gratuitous negative remarks about them: I

Don’t say: ”the deficiency of X’s approach is...” or “the problem of these papers is...” or “these papers suffer from X shortcomings”,... etc

I

Say: “We complement these approaches in a number of ways. First... Second... Third...”.

As an illustration, see Section 2.1. of “Does teacher truancy prevent students from learning? Evidence from Sub-Saharan Africa”. 38 / 71

3. Related literature

I

Note that you can also use the “Related literature” section to elaborate on some aspects of your paper.

I

For instance, you can provide more insight about why the answer to the research question is not obvious.

I

As an illustration, see Section 2.2. of “Does teacher truancy prevent students from learning? Evidence from Sub-Saharan Africa” (it provides a more detailed explanation about why teacher truancy may have an heterogeneous impact on educational achievements, conditional on students’ SES).

39 / 71

4. The data I

You must present the data that allow you to construct the variables that enter your analysis.

I

Divide this section into as many subsections as there are sets of variables: I

the subsection related to the dependent variable;

I

the subsection related to the (endogenous) explanatory variable;

I

the subsection related to the instrument (if relevant and in case the “empirical strategy” section precedes the “data” section);

I

the subsection related to the controls... etc

40 / 71

4. The data

I

When more than one source is available for a specific variable, you must justify why you use a source rather than the other.

I

For instance, when Miguel, Satyanath and Sergenti (2004, Journal of Political Economy) estimate the impact of economic growth on civil war onset, they have two datasets at their disposal to measure their dependent variable (i.e. civil war onset): I

Armed Conflict Data (ACD);

I

Correlates Of War (COW).

41 / 71

4. The data I

Here is their justification:

42 / 71

4. The data

I

Don’t forget to present and comment the descriptive statistics related to each set of variables.

I

These consist in providing the number of observations, the mean, as well as the standard deviation related to each of these variables (you can also display the minimum and the maximum values).

I

To get these statistics, use the following STATA command: tabstat X Y Z, stats(count mean sd min max) columns(statistics)

43 / 71

4. The data

I

See Section 3. but also Section 5.2. of “Does teacher truancy prevent students from learning? Evidence from Sub-Saharan Africa” for a complete example.

44 / 71

5. Empirical strategy

I

I

The purpose of this section is to: I

describe this strategy in details;

I

convince the reader that it is indeed a good one (i.e. it allows you to solve endogeneity problems).

For instance, if you rely on an instrumental variable approach, convince the reader that your instrument is valid (i.e. it has an impact on the dependent variable ONLY THROUGH its impact on the endogenous explanatory variable).

45 / 71

5. Empirical strategy

I

Don’t hesitate to rely on “visual” arguments.

I

For instance, to document the correlation between the instrument and the endogenous explanatory variable, you are encourage to present a scatterplot that presents the bivariate relationship between these variables.

I

To do so, use the following STATA command: graph twoway lfit x z||scatter x z, title("Correlation between x and z") xtitle("Variable x") ytitle("Variable z")

46 / 71

5. Empirical strategy

I

See Section 5.1. of “Does teacher truancy prevent students from learning? Evidence from Sub-Saharan Africa” for a complete example.

47 / 71

6. Results

I

First, you must explicitly define the regression equations that you estimate: I

write down these equations clearly, using the adequate notations and defining each variable precisely;

I

explain clearly what each of the main coefficients in your regression captures.

48 / 71

6. Results I

Second, the “Results” section should provide regression tables that must be SELF-EXPLANATORY.

I

“That is, a reader should be able to understand a table without reference to the text (and preferably without reference to other tables). The converse rule is that the text should stand by itself, even if a reader ignores the tables (Levine, undated, “David Levine’s Cheap Advice: How to Present Results”).

I

See Table 4 and Table 6 of “Does teacher truancy prevent students from learning? Evidence from Sub-Saharan Africa” for an illustration.

49 / 71

6. Results

I

In the Econometrics and Impact Evaluation course (Part III), we will explain how to get these tables from the estout STATA command.

50 / 71

6. Results

I

Third, the main results must be clearly derived from the regression tables in the “Results” section.

I

Focus on the economic magnitude of your critical coefficients, not just their statistical significance.

I

“Especially in large panel data sets even the tiniest of effects is “statistically significant.”” (Cochrane, 2005).

51 / 71

6. Results

I

See Section 4 and Section 5.3. of “Does teacher truancy prevent students from learning? Evidence from Sub-Saharan Africa” for a complete example.

52 / 71

7. Robustness checks

I

In this section, you must ensure that your findings cannot be due to a different story than the one you are telling.

I

For instance, you may identify an additional potential and non obvious competing channel (other than the one that involves the endogenous explanatory variable) through which your instrument can impact the dependent variable.

I

Show that your findings aren’t impacted if you control for this channel in your 2-SLS.

53 / 71

7. Robustness checks I

In case you cannot find alternative stories that could challenge your identification strategy (which is good), then proceed to softer robustness checks.

I

Is your main result sensitive to: I

alternative measures of your dependent variable?

I

alternative measures of your critical explanatory variables?

I

alternative measures of your instrument?

I

deleting outliers?

I

... etc

54 / 71

7. Robustness checks

I

See Section 6 of “Does teacher truancy prevent students from learning? Evidence from Sub-Saharan Africa” for a complete example.

55 / 71

8. Conclusion

I

The conclusion should be short (no more than 3 pages): I

start by summarizing your contribution;

I

then emphasize the real-world (policy) implications of your findings;

I

finally, point to avenues for future research (i.e.: how your paper could be extended). This can be the opportunity for you to acknowledge the limitations of your paper, but do this without being too negative about your own work.

56 / 71

8. Conclusion

I

See Section 7 of “Does teacher truancy prevent students from learning? Evidence from Sub-Saharan Africa” for a complete example.

57 / 71

9. References I

I

In the text: I

Spell out all the names of the authors you cite (do not use abbreviation like MSS for Miguel, Satyanath and Sergenti);

I

Indicate the year of publication of their work you refer to between parentheses.

In the “References” section: I

The list of references must be arranged alphabetically by author, complete and accurate;

I

A consistent style must be used regarding capitalization, italics, initials, and ordering of material.

58 / 71

9. References

I

See Section 8 of “Does teacher truancy prevent students from learning? Evidence from Sub-Saharan Africa” for an illustration.

59 / 71

10. Other writing tips

I

Every sentence should have a subject, verb and object;

I

Use “I” instead of the royal “we” whenever you write a sole-authored paper (which will be the case of your Master 2 thesis).

60 / 71

10. Other writing tips I

However, to avoid overwhelming the reader with “I”, also use your paper or elements of is as the subjects: I

“The objective of this paper is...” instead of “My objective is...”;

I

“This paper contributes to...” instead of “I contribute to...”;

I

“The paper proceeds as follows...” instead of “I organize this paper as follows...”;

I

“Table 4 presents OLS estimates...” instead of “I present OLS estimates in Table 4...”

I

... etc

61 / 71

10. Other writing tips

I

Use active tense whenever possible: “I estimate equation (1)...” instead of “Equation (1) is estimated...”;

I

Avoid an excessive use of “very”, “extremely”, “highly”... etc

I

Don’t start sentences with “This” unless you follow it with the associated noun: “This evidence indicates that...” instead of “This indicates that...”. This will make your writing clearer, more elegant and therefore more persuasive.

62 / 71

10. Other writing tips

I

I

To know when to create a footnote, follow Cochrane’s advices (2005): I

“Do you really want the reader to stop and read this? Then it should be in the text.”

I

“Do you think the average reader should not stop? Then delete the footnote.”

“Long lists of references, simple bits of algebra, or other documentation are good candidates for footnotes.”

63 / 71

10. Other writing tips

I

Check your paragraph breaks to make sure that each paragraph contains one (and only one) fully articulated idea.

I

Find a way to break up long paragraphs even if they contain a single idea.

I

Readers are subconsciously looking for open space. As a consequence, no paragraph should be longer than half a page (it is better if each full page has more than two paragraphs).

64 / 71

10. Other writing tips I

When writing, get inspiration from others’ work: I

get inspiration from the examples above mentioned;

I

get inspiration from economists known for being successful writers (use their papers as a template): I

Daron Acemoglu: http://econ-www.mit.edu/faculty/acemoglu

I

Nathan Nunn: http://www.economics.harvard.edu/faculty/nunn

I

Andrei Shleifer: http://www.economics.harvard.edu/faculty/shleifer

I

... etc

65 / 71

10. Other writing tips

I

By “getting inspiration”, we mean “imitating their words and phrases” that you will modify to suit your research purpose.

I

This has therefore nothing to do with plagiarism that concerns ideas, not style.

66 / 71

Conclusion

I

Writing a research paper is a process of discovery, first and foremost for the author himself/herself (it reveals gaps in the argument).

I

So start writing your Master 2 thesis as soon as your research proposal is “endorsed” by your supervisor.

I

Sketch briefly the content of each section.

I

Then generate the text.

67 / 71

Conclusion

I

While writing, follow Creedy’s advice (2001):

“Re-read as you go along. In particular, before turning to a new paragraph, read the previous one. Before starting a new writing session, re-read the previous work. This will help to improve continuity. Regularly check the linkages between sections.”

68 / 71

Conclusion

I

Once you have a first draft, you will have to recognize that it won’t be the last: “all drafts must be edited and polished many times, paying close attention to detail as well as the overall shape and flow of the argument” (Creedy, 2001).

I

Be extremely critical whenever you read your work: put yourself in your reader’s position, and try to think in advance of potential criticisms.

69 / 71

Conclusion

I

This revision process takes time BUT organize yourself such that you will be done by the end of May (attach a time schedule to the writing of your M2 thesis).

I

“This will allow you to leave the paper alone for a while and then give it a final polish after returning to it refreshed. You will be surprised by how many small but significant improvements can be made” (Creedy, 2001).

70 / 71

Conclusion

I

I

Your Master 2 thesis will be assessed based on whether you complied with the requirements contained in the two Research Methodology classes: I

Defining a good research project.

I

Writing a good research paper.

So do not neglect any of these requirements if you are yearning for a good grade!

71 / 71