Principles of Mathematical Economics - Lakehead University

Principles of Mathematical Economics - Lakehead University

Principles of Mathematical Economics Kam Yu Kam Yu Lakehead University Thunder Bay, Ontario Canada Phone +1-807-343-8229 Email: [email protected]

2MB Sizes 0 Downloads 22 Views

Principles of Mathematical Economics

Kam Yu

Kam Yu Lakehead University Thunder Bay, Ontario Canada Phone +1-807-343-8229 Email: [email protected]

c 2015 by Kam Yu Copyright All rights reserved.

Preface Every year a first-year student, out of curiosity or because of some program requirements, takes principles of economics. She is surprised to learn that a subject in social science can be studied in such a logical and scientific manner. After getting a good grade, she decides to major in economics. The advisor in the department told the student that quantitative analysis is very important in the study of economics. A degree in economics, however, is a liberal arts degree, which provides a well-rounded general education and a good preparation for professional schools. Many economics majors, the advisor said, achieve high scores in LSAT, an entrance examination for the faculty of law. The student diligently select courses in calculus, linear algebra, probability and statistics, and econometrics in her curriculum and does well in those courses. After graduation she decides to pursue graduate study in economics. To the student’s surprise, the verbal and graphical analysis she was trained to solve economic problems somehow all but disappear. Instead, most of the graduate textbooks contain materials that more or less resemble applied mathematics. It seems that there is a wide gap between the general liberal arts degree and graduate studies in economics. The objective of this book is to fill such a gap. It is written for a one-semester course as an introduction to mathematical economics for first year graduate and senior undergraduate students. The materials assume the students have backgrounds in college calculus and linear algebra. A course in mathematical economics at the undergraduate level is helpful but not necessary. Chapters 1 through 5 aim to build up the students’ skills in formal proof, axiomatic treatment of linear algebra, and elementary vector differentiation. Chapters 6 and 7 present the basic tools needed for microeconomic analysis. I have covered much of the materials up to chapter 7 to such a class for many years. Due to the time constraint of a short semester, many proofs of the theorems are omitted. For example, although the inverse function theorem and the implicit function theorem are important tools in economic analysis, iii

Preface experience has shown that the marginal benefits of going through the proofs in class are not worth the distraction. Therefore throughout the text I have included notes that point to references if the students want to explore a bit more about particular topics. For the above reasons, the book is not meant to be a manual or a reference in mathematical economics. My focus is to provide a quick training to graduate students in one semester to acquire a somewhat elusive quality called mathematical maturity. Greg Mankiw advises his students to take as many courses in mathematics “until it hurts.” Different students, of course, have different levels of tolerance. There were occasional moments of delight, when a student exclaimed that the proof of a theorem was simply beautiful. Nevertheless, my apology if this book really hurts. There are two bonus chapters. Most graduate level econometrics courses assume the students to have some training in mathematical probability and statistics. Chapter 8 provides a quick introduction or review to probability theory. It is, however, by no mean a substitute for the formal training. Chapter 9 is an introduction to dynamic modelling, which I used as a supplementary reading for a course in macroeconomic theory. Both chapters sometimes refer to concepts covered in the core chapters. Part of the book was written when I was visiting Vancouver, British Columbia. On one summer day I joined a tour with my friend to Whistler. While the tour guide was introducing the Coast Mountains and other points of interest, my mind at one moment drifted back to the time when I was new to the province. I saw afresh how beautiful the country was. In writing this book, there were moments that I thought about the materials from a student’s perspective, and rediscover the beauty of mathematics. I hope you enjoy the journey. Last but not least, I would like to thank my friends Eric, Erwin, and Renee, who continue to give me supports, in good time and bad.

iv

Contents Preface

iii

Table of Contents

v

List of Figures

xii

List of Tables

xiii

1 Logic and Proof 1.1 Logical Statements . . . . . . . . . . . . 1.2 Logical Connectives . . . . . . . . . . . . 1.3 Quantified Statements . . . . . . . . . . 1.4 Mathematical Proof . . . . . . . . . . . . 1.4.1 Direct Proof of Implications . . . 1.4.2 Proof by Contrapositives . . . . . 1.4.3 Proof by Contradiction . . . . . . 1.4.4 Proving Implications with Cases . 1.4.5 Proving Equivalent Statements . 1.4.6 Proving Quantitative Statements 1.5 Strategies in Mathematical Proof . . . . 1.6 Exercises . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . .

2 Sets and Relations 2.1 Basic Notations . . . . . . . . . . . . . . . 2.2 Set Operations . . . . . . . . . . . . . . . 2.3 Binary Relations . . . . . . . . . . . . . . 2.4 Well-Ordering and Mathematical Induction 2.5 Functions . . . . . . . . . . . . . . . . . . v

. . . . . . . . . . . .

. . . . .

. . . . . . . . . . . .

. . . . .

. . . . . . . . . . . .

. . . . .

. . . . . . . . . . . .

. . . . .

. . . . . . . . . . . .

. . . . .

. . . . . . . . . . . .

. . . . .

. . . . . . . . . . . .

. . . . .

. . . . . . . . . . . .

. . . . .

. . . . . . . . . . . .

. . . . .

. . . . . . . . . . . .

. . . . .

. . . . . . . . . . . .

. . . . .

. . . . . . . . . . . .

. . . . .

. . . . . . . . . . . .

. . . . .

. . . . . . . . . . . .

. . . . .

. . . . . . . . . . . .

1 1 2 4 6 7 8 9 11 12 12 14 15

. . . . .

21 21 24 29 34 36

CONTENTS 2.6

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3 Basic Topology 3.1 Introduction to Metric Space . . . . . . . . 3.1.1 Topological Space . . . . . . . . . . 3.1.2 Metric Space . . . . . . . . . . . . 3.1.3 Definitions in Metric Space . . . . . 3.1.4 Some Examples . . . . . . . . . . . 3.1.5 Sequences . . . . . . . . . . . . . . 3.1.6 Series in R . . . . . . . . . . . . . . 3.2 Continuous Functions . . . . . . . . . . . . 3.2.1 Properties of Continuous Functions 3.2.2 Semicontinuity . . . . . . . . . . . 3.2.3 Uniform Continuity . . . . . . . . . 3.2.4 Fixed Point Theorems . . . . . . . 3.3 Sequences and Series of Functions . . . . . 3.3.1 Pointwise Convergence . . . . . . . 3.3.2 Uniform Convergence . . . . . . . . 3.4 Exercises . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

4 Linear Algebra 4.1 Vector Spaces . . . . . . . . . . . . . . . . . . . . 4.2 Basic Properties of Vector Spaces . . . . . . . . . 4.3 Linear Transformations . . . . . . . . . . . . . . . 4.3.1 Introduction . . . . . . . . . . . . . . . . . 4.3.2 Matrix Representations of Transformations 4.3.3 Linear Functionals . . . . . . . . . . . . . 4.3.4 Multilinear Transformations . . . . . . . . 4.4 Inner Product Spaces . . . . . . . . . . . . . . . . 4.5 Inequalities . . . . . . . . . . . . . . . . . . . . . 4.5.1 Cauchy-Schwarz Inequality . . . . . . . . . 4.5.2 Triangular Inequality . . . . . . . . . . . . 4.6 Determinants . . . . . . . . . . . . . . . . . . . . 4.7 Inverse Transformations . . . . . . . . . . . . . . 4.7.1 Systems of Linear Equations . . . . . . . . 4.7.2 Inverse of a Matrix . . . . . . . . . . . . . 4.7.3 Cramer’s Rule . . . . . . . . . . . . . . . . vi

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

42

. . . . . . . . . . . . . . . .

49 49 49 49 50 51 52 54 58 58 60 61 62 63 63 64 67

. . . . . . . . . . . . . . . .

75 75 80 86 86 87 89 91 91 93 93 94 95 96 96 97 99

CONTENTS 4.7.4 Properties of Inverses . . . . . . . . . . . . 4.8 Eigenvectors and Eigenvalues of Linear Operators 4.8.1 Definitions . . . . . . . . . . . . . . . . . . 4.8.2 Symmetric Operators . . . . . . . . . . . . 4.9 Quadratic Forms . . . . . . . . . . . . . . . . . . 4.9.1 Positive Definite Matrices . . . . . . . . . 4.9.2 Negative Definite Matrices . . . . . . . . . 4.10 Exercises . . . . . . . . . . . . . . . . . . . . . . . 5 Vector Calculus 5.1 Vector Differentiation . . . . . . . . 5.1.1 Introduction . . . . . . . . . 5.1.2 Functional Analysis . . . . . 5.2 Approximation of Functions . . . . 5.2.1 Taylor Series . . . . . . . . 5.2.2 Logarithmic Approximation 5.2.3 Log-Linear Approximation . 5.3 Inverse Functions . . . . . . . . . . 5.4 Exercises . . . . . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

6 Convex Analysis 6.1 Basic Definitions . . . . . . . . . . . . . 6.2 Convex Sets . . . . . . . . . . . . . . . . 6.3 Affine Sets . . . . . . . . . . . . . . . . . 6.3.1 Definitions . . . . . . . . . . . . . 6.3.2 Hyperplanes and Half-Spaces . . 6.4 Convex Functions . . . . . . . . . . . . . 6.4.1 Convex and Concave Functions . 6.4.2 Quasi-Convex and Quasi-Concave 6.4.3 Increasing Functions on Rn+ . . . 6.5 Homogeneous Functions . . . . . . . . . 6.6 Separating Hyperplanes . . . . . . . . . 6.7 Exercises . . . . . . . . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . .

. . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Functions . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . .

. . . . . . . . .

. . . . . . . . . . . .

. . . . . . . .

. . . . . . . . .

. . . . . . . . . . . .

. . . . . . . .

. . . . . . . . .

. . . . . . . . . . . .

. . . . . . . .

. . . . . . . . .

. . . . . . . . . . . .

. . . . . . . .

. . . . . . . . .

. . . . . . . . . . . .

. . . . . . . .

. . . . . . . . .

. . . . . . . . . . . .

. . . . . . . .

. . . . . . . . .

. . . . . . . . . . . .

. . . . . . . .

. . . . . . . . .

. . . . . . . . . . . .

. . . . . . . .

. . . . . . . . .

. . . . . . . . . . . .

. . . . . . . .

99 100 100 102 103 104 105 106

. . . . . . . . .

115 115 115 118 121 121 123 124 125 128

. . . . . . . . . . . .

135 135 136 138 138 139 140 140 143 144 145 147 149

7 Optimization 153 7.1 Unconstrained Optimization . . . . . . . . . . . . . . . . . . . . . . . 153 7.2 Equality-Constrained Optimization . . . . . . . . . . . . . . . . . . . 155 vii

CONTENTS 7.3 7.4

Inequality-Constrained Optimization . . . . . . . . . . . . . . . . . . 161 Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

8 Probability 8.1 Basic Definitions . . . . . . . . . . . . . . . . 8.2 Conditional Probability . . . . . . . . . . . . . 8.3 Probability Distributions . . . . . . . . . . . . 8.4 Mathematical Expectations . . . . . . . . . . 8.5 Some Common Probability Distributions . . . 8.5.1 Binomial Distribution . . . . . . . . . 8.5.2 Geometric Distribution . . . . . . . . . 8.5.3 Gamma and Chi-Square Distributions . 8.5.4 Normal Distribution . . . . . . . . . . 8.6 Multivariate Distributions . . . . . . . . . . . 8.7 Stochastic Processes . . . . . . . . . . . . . . 8.7.1 Martingales . . . . . . . . . . . . . . . 8.7.2 Moving Average Models . . . . . . . . 8.7.3 Autoregressive Models . . . . . . . . . 8.8 Exercises . . . . . . . . . . . . . . . . . . . . . 9 Dynamic Modelling 9.1 First-Order Difference Equation . 9.1.1 Constant Coefficients . . . 9.1.2 Non-constant Coefficients 9.2 Dynamical Systems . . . . . . . . 9.3 Dynamic Programming . . . . . . 9.3.1 Basic Theory . . . . . . . 9.3.2 Recipe . . . . . . . . . . . 9.3.3 Linear Quadratic Models . 9.3.4 Finite Horizon Problems . 9.4 Autonomous Dynamical Systems 9.4.1 Linear Systems . . . . . . 9.4.2 Nonlinear Systems . . . . 9.4.3 Phase Diagrams . . . . . . 9.4.4 Stochastic Processes: Once 9.5 Exercise . . . . . . . . . . . . . . viii

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . More with Feeling . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

173 173 177 180 182 186 186 187 187 188 190 193 193 193 194 195

. . . . . . . . . . . . . . .

199 199 199 202 205 207 207 209 212 213 215 215 217 219 222 227

CONTENTS Bibliography

233

ix

CONTENTS

x

List of Figures 2.1 2.2

Venn Diagrams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Epigraph of a Functional . . . . . . . . . . . . . . . . . . . . . . . . .

25 41

3.1 3.2

The Exponential Function y = ex . . . . . . . . . . . . . . . . . . . . . Semicontinuous Functions . . . . . . . . . . . . . . . . . . . . . . . .

56 61

4.1 4.2 4.3

Vector Addition in R2 . . . . . . . . . . . . . . . . . . . . . . . . . . The Rotation Transformation . . . . . . . . . . . . . . . . . . . . . . An Orthogonal Projection from R3 onto R2 . . . . . . . . . . . . . . .

77 89 93

5.1 5.2 5.3 5.4

Gradient of a Functional . . . . . . Taylor Approximations of y = sin x Logarithmic Approximation . . . . Log-Linear Approximation . . . . .

6.1 6.2 6.3

Convex and Non-Convex Sets . . . . . . . . . . . . . . . . . . . . . . 137 Constructing a Convex Function from a Convex Set . . . . . . . . . . 142 Convex and Concave Functions . . . . . . . . . . . . . . . . . . . . . 145

7.1 7.2

Graph of the saddle z = x2 − y 2 . . . . . . . . . . . . . . . . . . . . . 155 A Cusp Manifold . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168

8.1

Standard Normal Distribution . . . . . . . . . . . . . . . . . . . . . . 189

9.1 9.2 9.3 9.4 9.5 9.6

First-Order Difference Equation . . . Convergence of First-Order Difference Price/Earning Ratio of U.S. Stocks . A Linear Dynamical System . . . . . Phase Diagram of a Linear System . Phase Diagram of the Ramsey Model xi

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . . . . Equation . . . . . . . . . . . . . . . . . . . . . . . .

. . . .

. . . . . .

. . . .

. . . . . .

. . . .

. . . . . .

. . . .

. . . . . .

. . . .

. . . . . .

. . . .

. . . . . .

. . . .

. . . . . .

. . . .

. . . . . .

. . . .

. . . . . .

. . . .

. . . . . .

. . . .

. . . . . .

. . . .

. . . . . .

120 122 123 125

201 201 204 218 220 222

LIST OF FIGURES 9.7

The Poverty Trap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230

xii

List of Tables 1.1

Some Useful Tautologies . . . . . . . . . . . . . . . . . . . . . . . . .

5.1

Some Common Functionals . . . . . . . . . . . . . . . . . . . . . . . . 119

7.1

Values of (−1)j |Bj | . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

9.1

Dimensions of Derivatives . . . . . . . . . . . . . . . . . . . . . . . . 209

xiii

14

LIST OF TABLES

xiv

Chapter 1 Logic and Proof 1.1

Logical Statements

Most languages follow some grammatical rules in order to convey ideas clearly. Mathematics written in English of course has to follow these rules. Moreover, the ideas have to obey logical rules to be meaningful. In this chapter we review the commonly used logical rules in mathematics. We start with the idea of a statement, which is a sentence that a truth value can be assigned. The truth value is either true (T) or false (F), but not both. Example 1.1. (a) Aurorae are formed when solar particles interact with the Earth’s magnetic field. (b) The cardinality of the set of real numbers is bigger than that of the rational numbers. (c) All prime numbers are odd numbers. (d) Every child should be entitled to a good education. (e) x2 + 1 = 0. The first three sentences are statements. Statement (a) is true based on inductive reasoning. That is, it can be verified by repeated empirical testings and observations, which is the foundation of scientific inquiries. Statement (b) is true because of deductive reasoning. Mathematical knowledge is derived from agreements among mathematicians on assumptions called axioms. Theorems and propositions are derived from the axioms by logical rules. Therefore pure mathematics in principle is not science since it does not rely on empirical observations. Statement (c) is false 1

CHAPTER 1. LOGIC AND PROOF since 2 is prime but even. The sentence in (d) is a moral declaration. It belongs to the discussion in ethics and cannot be in principle assigned the value of true or false. The last sentence is not a statement because there are some numbers which satisfy the equation, while others do not. We shall later impose more structure on the sentence to make it a statement. Notice that we do not have to know the truth value before deciding if a sentence is a statement. All we need is that a truth value can in principle be determined.

1.2

Logical Connectives

Logical connectives are operations on statements that change their truth values or combine them to form new statements. If p is a statement, the negation of p, written as ∼ p, has the opposite truth value of p. That is, if p is true, then ∼ p is false. If p is false, then ∼ p is true. The operation can be illustrated by a truth table: p ∼p T F F T For example, if p is “Today is Sunday”, then ∼ p means “Today is not Sunday”. You should convince yourself that ∼ (∼ p) has the same truth value as p. If p and q are two statements, the conjunction of p and q, also called “p and q” and denoted by p ∧ q, is a statement that is true when both p and q are true. Otherwise it is false. The following truth table lists the four cases: p∧q T F F F

p q T T T F F T F F

The disjunction of p and q, also called “p or q” and denoted by p ∨ q, is a statement that is true when at least one of p and q is true. The truth table is p∨q T T T F

p q T T T F F T F F 2

1.2. LOGICAL CONNECTIVES Our definition of disjunction is sometimes called “inclusive or”. In other applications such as computer science, an “exclusive or” means that the disjunction is true if only one of p and q is true, but not both. Notice that the order of p and q is not important in conjunction and disjunction. That is, “p and q” is the same as “q and p”. This is not true for an implication, which is written as “If p then q”, and denoted by p ⇒ q. The statement p is called the hypothesis and q the conclusion. An implication is also called a conditional statement. The rule is defined in the following truth table: p q p⇒q T T T T F F F T T F F T That is, an implication is false when the hypothesis is true but the conclusion is false. When the hypothesis is false, the implication is true whether the conclusion is true or false. Example 1.2. Consider the implication “If I am the queen of England, then I would make you a knight.” Since the hypothesis is false (I am not the queen), whether I keep my promise or not is irrelevant. Therefore I do not make a false statement. Implication is one of the most common forms of logical statements. There are many different ways to express the same idea. The following statements all mean “If p then q.” • p implies q. • q whenever p. • q if p. • p only if q. • q is the necessary condition of p. • p is a sufficient condition for q. Given an implication p ⇒ q, it is sometimes useful to define the following induced conditional statements: 3

CHAPTER 1. LOGIC AND PROOF Converse: q ⇒ p. Inverse: (∼ p) ⇒ (∼ q). Contrapositive: (∼ q) ⇒ (∼ p). Two statements p and q are equivalent statements if p implies q and q implies p. That is, (p ⇒ q) ∧ (q ⇒ p), which has a shorthand notation p ⇔ q. As the following truth table shows, this happens when p and q have the same truth value. p q T T T F F T F F

p⇒q T F T T

q⇒p T T F T

p⇔q T F F T

In other words, p is equivalent to q when p implies q and the converse is also true. Often we write equivalent statements as “p if and only if q”. All definitions are equivalent statements. However, the following example illustrates that we use “if” in definitions when we actually means “if and only if”. Example 1.3. An integer n is odd if it can be expressed as n = 2m + 1 where m is an integer. Here the statements p and q are • p: n is an odd integer. • q: n = 2m + 1 where m is an integer. Intuitively, equivalent statements have the same meaning and therefore can be used as alternative definitions. Often we call finding equivalent statements characterization of the original statement.

1.3

Quantified Statements

In section 1.1 we claim that x2 + 1 = 0 is not a statement. We can turn it into a statement by adding a quantifier. For instance, “There exists a real number x such that x2 + 1 = 0” is an example of existential statement. In logical symbols, it reads ∃ x ∈ R 3 x2 + 1 = 0. 4

1.3. QUANTIFIED STATEMENTS The symbol ∃ above means “there exists” and is formally called an existential quantifier. The symbol ∈ means “in” or “belongs to”, and R denotes the set of real numbers (we shall define them in details in the next chapter). The symbol 3 reads “such that”, which is sometimes abbreviated as “s.t.” The statement means that there is at least one real number x that satisfies the equation that follows, which in this case is false. The reason is that no real number satisfies that equation √ x2 + 1 = 0, since it implies that x = ± −1. We can, however, make the statement true by expanding the choice of x to include the set of complex numbers, C: ∃ x ∈ C 3 x2 + 1 = 0.

In daily conversations, we do not use such a formal language. Instead of saying “there exists a bird such that it can swim,” we say “some birds can swim.” Sometimes we want to assign a condition or property that is shared by all members of a group. In that case we need the universal quantifier. For example, we want to express that fact that the squared values of all real numbers is equal to or greater than zero. In logical symbols, it becomes ∀ x ∈ R, x2 ≥ 0. The symbol ∀ denotes “for all” or “for every”. The statement requires that all members of the set R satisfy the condition that follows. In other words, if we can find even one real number x such that x2 < 0, then the statement is false. For example, the statement ∀ x ∈ R, x2 > 0 (1.1) is false since when x = 0, x2 = 0 instead of x2 > 0. In general, if p(x) is a statement that its truth value depends on a variable x, a universal statement is in the form ∀ x, p(x). The above statement is true if every x in the context of our discussion makes p(x) true. On the other hand, if there is one x such that p(x) is false, the universal statement is false. It follows that the negation of a universal statement, ∼ (∀ x, p(x)) is equivalent to ∃ x 3 ∼ p(x). 5

CHAPTER 1. LOGIC AND PROOF For example, the negation of the statement in (1.1) is ∃ x ∈ R 3 x2 ≤ 0, which reads “there exists a real number such that its square is less than or equal to zero.” The general form of an existential statement is ∃ x 3 p(x).

(1.2)

For the statement to be true, we need to find at least one x such that p(x) is true. On the other hand, the statement is false if every x in question makes p(x) a false statement. Therefore the negation of the existential statement in (1.2) is equivalent to ∀ x, ∼ p(x). When we write quantified statements involving more than one quantifiers, their order is important because it can affect the meaning of the statement. Consider the following statements under the context of real numbers: ∃ x 3 ∀ y, x > y,

(1.3)

∀ y, ∃ x 3 x > y.

(1.4)

and

Statement (1.3) means that we can find a real number that is greater than any real number, which is false. On the other hand, statement (1.4) asserts that for any given real number, we can find a greater one.

1.4

Mathematical Proof

Consider the conditional statement “if the money supply in an economy increases faster than its growth in output, then inflation will follow.” Economists spend a lot of time debating whether the statement is always true, sometimes true, or always false. In science, a “law” is a statement that has been conforming to empirical evidence. A general rule is that a scientific law is accepted as true until new observations show otherwise. The world of mathematics is more certain. A theorem derived from universally accepted axioms is always true. Such a statement is called a tautology. 6

1.4. MATHEMATICAL PROOF Therefore, in a sense, mathematics is the study of tautologies using logical rules. For example, the theorems in the book Elements of Geometry, written by the Greek mathematician Euclid in 300 B.C., are still true under the assumption of his postulates and axioms. In this section we discuss some useful techniques in mathematical proofs. Interested readers can consult books such as Beck and Geophegan (2010) or Gerstein (2012) for more detailed discussions. Before discussing proof, we first define some common terms in mathematics. An axiom is a statement assumed to be true in a mathematical system. For example, in Euclid’s Elements, one of the axioms (which he calls a common notion) is Things that are equal to the same thing are also equal to one another. This axiom is called “transitivity of equality,” which means that if a = b and b = c, then a = c. Axioms and definitions are the basic rules of a mathematical system They are analogous to the rules of a chess game — no question asked.1 A proposition, conjectures or postulate is a statement which contains ideas that can be true or false. One of the most famous conjectures is Fermat’s Last Theorem, which states that no three positive integers a, b, and c can satisfy the equation an + bn = cn for any integer value of n greater than two. It was proposed by the French mathematician Pierre de Fermat in the seventeenth century. When it was finally proven by Andrew Wiles in 1995, it became a theorem, a logically proven true statement. A lemma is a proven true statement that is useful in proving other results, while a corollary is a true statement that is a consequence of a theorem.

1.4.1

Direct Proof of Implications

Recall in section 1.2 that a conditional statement is true when • the hypothesis and the conclusion are both true, or • the hypothesis is false, regardless the truth value of the conclusion. In the second case, the implication is true by definition so there is nothing to prove. Therefore to prove that an implication is true directly, we need to assume that the hypothesis p is true and then “show” that the conclusion is also true. But what do we mean by “showing”? The following equivalent statement is helpful: (p ⇒ q) ⇔ (p ⇒ p1 ⇒ p2 ⇒ · · · ⇒ q2 ⇒ q1 ⇒ q). 1

For a more detailed discussion of axioms and proof see Devlin (2002).

7

CHAPTER 1. LOGIC AND PROOF That is, we can take a number of intermediate steps between p and q, and in each step we arrive at a new true statement (p1 , p2 , . . . , q2 , q1 , etc.). We know that the intermediate conditional statements are true by invoking axioms, definitions, and proven results. For example, suppose we want to prove that if n is a positive integer, then 2 n + 3n + 8 is an even number. The key ingredients of this conditional statement are • Hypothesis p: n is a positive integer, • Conclusion q: n2 + 3n + 8 is an even number. For a direct proof, the first step is to assume that p is true, that is, n is a positive integer. It is not obvious at this stage how we can arrive at the conclusion q. One strategy is to do some backward thinking about what statement would implies q, that is, what is q1 ? In this case the definition of an even number is helpful. An even number can be expressed as 2k, where k is an integer. Therefore somehow we must be able to express n2 + 3n + 8 as 2k. Another useful property is that (n + 1)(n + 2) is even since one of n + 1 or n + 2 is even and the product of an even number with any integer is even. Now we can write down the formal proof as follows. Proof. Suppose that n is a positive integer. Then (n + 1)(n + 2) = 2m is even since one of n + 1 or n + 2 is even and the product of an even number with any integer is even. Now n2 + 3n + 8 = n2 + 3n + 2 + 6 = (n + 1)(n + 2) + 6 = 2m + 2(3) = 2(m + 3) = 2k, where m and k = m + 3 are integers. Therefore n2 + 3n + 8 is even. You should be able to write down the intermediate statements in the above proof.

1.4.2

Proof by Contrapositives

Sometimes a direct proof of a conditional statement can be too complicated or too difficult. A very useful alternative method is by proving the contrapositive instead. 8

1.4. MATHEMATICAL PROOF The method depends on the following equivalent statements (see exercise 10): (p ⇒ q) ⇔ [(∼ q) ⇒ (∼ p)]. Therefore we proceed by assuming the negation of the conclusion is true and show that negation of the hypothesis is true. Example 1.4. Suppose that a < b and f (x) is a real-valued function. Prove that if Z

b

f (x) dx 6= 0, a

then there exists an x between a and b such that f (x) 6= 0. Here the hypothesis and the conclusion are Rb • p : a f (x) dx 6= 0, • q: there exists an x between a and b such that f (x) 6= 0, and we need to prove p ⇒ q. You can try the direct proof but the contrapositive of the statement provides an easy way out. The negations of p and q are Rb • ∼ p : a f (x) dx = 0, • ∼ q: For all x between a and b, f (x) = 0. Now we proceed to show that (∼ q) ⇒ (∼ p). Proof. Suppose that a < b and f (x) is a real-valued function. If f (x) = 0 for all x Rb between a and b, then by the definition of integration a f (x) dx = 0.

1.4.3

Proof by Contradiction

If both the direct method and the contrapositive fail to give a manageable proof of an implication, there is another alternative called proof by contradiction. The technique is based on the following equivalent statement. (p ⇒ q) ⇔ [p ∧ (∼ q) ⇒ c], where c is called a contradiction, defined as a statement that is always false. Therefore, instead of proving the implication directly, we assume that the hypothesis is 9

CHAPTER 1. LOGIC AND PROOF true but the conclusion is false. Then we show that this implies a statement that is known to be false or contradicts our assumption. Example 1.5. Prove that if x is rational and y is irrational, then x + y is irrational. Recall that a number x is rational if it can be expressed as ratio of two integers, that is, x = a/b, where a and b are integers and b 6= 0. Otherwise it is called irrational. Here we want to prove the conditional statement p ⇒ q where • p: x is rational and y is irrational. • q: x + y is irrational. We can prove this by contradiction, that is, we show that (p ∧ ∼ q) ⇒ c. Proof. Suppose that x is rational and y is irrational but x+y is rational. By definition x = a/b and x + y = c/d, where a, b, c, d are integers and b, d 6= 0. It follows that y=

c a bc − ad − = . d b bd

Since bc − ad and bd are integers and bd = 6 0, y is rational. This contradicts our assumption that y is irrational. Therefore x + y is irrational. Another case of proof by contradiction is the simple equivalent statement p ⇔ (∼ p ⇒ c). That is, instead of showing directly the statement p is true, we can show that its negation implies a contradiction. Example 1.6. The number



2 is irrational.

√ √ Proof. Assume on the contrary that 2 is rational. Hence 2 = m/n for some √ integers m and n with no common factors. It follows that m = 2n so that m2 = 2n2 . This means that m2 and therefore m are even integers. Writing m = 2k for some integer k, we have 2n2 = m2 = 4k 2 , or n2 = 2k 2 . This implies that n is also even, which contradicts our assumption that m and n have no common factors. 10

1.4. MATHEMATICAL PROOF

1.4.4

Proving Implications with Cases

Many conditional statements are in the form p ⇒ (q ∨ r). Instead of a direct proof, the following equivalent statement is very useful: [p ⇒ (q ∨ r)] ⇔ [(p ∧ (∼ q)) ⇒ r]. Of course the role of the statements q and r are symmetric. That is, we can show (p ∧ (∼ r)) ⇒ q instead. Example 1.7. Suppose that x is a real number. If 2x = x2 , then x = 0 or x = 2. Proof. Suppose that 2x = x2 and x 6= 0. Dividing both side of the equation by x gives x = 2. Another common conditional statement with case is in the form (p ∨ q) ⇒ r. That is, the hypothesis can be divided into two cases. For example, an integer is either odd or even. Similarly, a real number x is either negative or x ≥ 0. Or it is rational or irrational. A useful tool in proving this kind of statement is by using the following equivalent statement. [(p ∨ q) ⇒ r] ⇔ [(p ⇒ r) ∧ (q ⇒ r)]. That is, we show that both cases ensure the conclusion r. The following example employs this technique at the end of the proof. Example 1.8. Show that there exists irrational numbers x and y such that xy is rational. √ Proof. We have established in example 1.6 above that 2 is irrational. But we observe that  √ √2 √ 2 √ (√2√2) √ 2 2 = 2 = 2 = 2, √ √2 which is rational. Now we have two cases. First, if 2 is rational, then we let √ √ √2 √ √2 √ x = y = 2. Second, if 2 is irrational, then we let x = 2 and y = 2 as required. √ √2 Notice that in the above example we do not know whether 2 is rational or irrational. But in both cases we can find two irrational numbers to give the results. 11

CHAPTER 1. LOGIC AND PROOF

1.4.5

Proving Equivalent Statements

Recall that the general form of an equivalent statement is p ⇔ q, which is defined by (p ⇒ q) ∧ (q ⇒ p). Therefore the proof of the statement involves two parts. First we prove that p ⇒ q. Then we proof its converse q ⇒ p. It is not necessary to use the same technique for both parts. For example, it is perfectly fine to prove the first implication by the direct method and the converse by using its contrapositive. The following example illustrates this point. Example 1.9. Suppose that n is an integer. Prove that n is odd if and only if n2 is odd. Proof. In the first part we show that n is odd implies that n2 is odd. By definition of an odd number, n = 2k + 1, for some integer k. Therefore n2 = (2k + 1)2 = 4k 2 + 4k + 1 = 2(2k 2 + 2k) + 1. Since 2k 2 + 2k is also an integer, n2 is an odd number. This complete the proof of the first part. In the second part, we prove the converse by proving the contrapositive, which states that if n is even, then n2 is even. So suppose that n = 2k for some integer k. Then n2 = 4k 2 = 2(2k 2 ), which is even.

1.4.6

Proving Quantitative Statements

A universal statement proclaims that all members of a set under discussion satisfy a certain property. For example, the statement “all birds can fly” means that if we pick an arbitrary animal classified as a bird, then it must be able to fly. Of course the statement is false as there are indeed flightless birds. Therefore, in proving a universal statement ∀x, p(x), we need to pick an arbitrary member x in the set and show that p(x) is true. To disprove a universal statement, we have to find a counter-example. That is, if there exists at least an x such that ∼ p(x), then the universal statement is false. On the other hand, to prove an existential statement such as “some birds can fly,” we need to find at least one bird in the sky. To disprove an existential statement ∃x 3 p(x), we need to show that p(x) is false for all x in question. Example 1.10. Prove that for all real number x, there exists a number y such that y > x. 12

1.4. MATHEMATICAL PROOF Proof. Suppose that x is any arbitrary real number. Let y = x + 1. Then y > x as required. Example 1.11. Prove that the square of all odd integers can be expressed as 8k + 1 for some integer k. Proof. Let n be any odd integer. Then n = 2m + 1 for some integer m. Now n2 = (2m + 1)2 = 4m2 + 4m + 1 = 4m(m + 1) + 1. Since either m or m + 1 is even, 4m(m + 1) = 8k where k is an integer. So we conclude that n2 = 8k + 1. Example 1.12. Prove that there exists a real number x such that for all y and z, if y > z, then y > x + z. Proof. Choose x = 0. Then for all real numbers y and z such that y > z, y > 0 + z. Example 1.13. Suppose that f (n) = 1 − 1/n for n = 1, 2, 3, . . .. For all  > 0, show that there exists a positive integer N such that for all n > N , |f (n) − 1| < .

(1.5)

We need to take any arbitrary value of a positive  and then find an N that all n > N satisfy the inequality in (1.5). Now |f (n) − 1| = |1 − 1/n − 1| = | − 1/n| = 1/n. Therefore we need to have 1/n < , or n > 1/. Hence we can let N = d1/e, where dxe is the ceiling function, defined as the smallest integer greater than the number x. For example, if  = 0.07, then N = d1/0.07e = d14.29e = 15. Here is the formal proof. Proof. Given any  > 0, let N = d1/e. Then for all n > N , |f (n) − 1| = |1 − 1/n − 1| = | − 1/n| = 1/n < 1/N ≤  as required. 13

CHAPTER 1. LOGIC AND PROOF

Table 1.1: Some Useful Tautologies (a) ∼ (p ∧ q) ⇔ (∼ p) ∨ (∼ q) (b) ∼ (p ∨ q) ⇔ (∼ p) ∧ (∼ q) (c) ∼ [∀ x, p(x)] ⇔ [∃ x 3 ∼ p(x)] (d) ∼ [∃x 3 p(x)] ⇔ [∀ x, ∼ p(x)] (e) (p ⇒ q) ⇔ (p ⇒ p1 ⇒ p2 ⇒ · · · ⇒ q2 ⇒ q1 ⇒ q) (f) (p ⇒ q) ⇔ [(∼ q) ⇒ (∼ p)] (g) (p ⇒ q) ⇔ [p ∧ (∼ q) ⇒ c] (h) p ⇔ (∼ p ⇒ c) (i) [p ⇒ (q ∨ r)] ⇔ [(p ∧ (∼ q)) ⇒ r] (j) [(p ∨ q) ⇒ r] ⇔ [(p ⇒ r) ∧ (q ⇒ r)] (k) [p ∧ (p ⇒ q)] ⇔ q (l) ∼ (p ⇒ q) ⇔ [p ∧ (∼ q)] (m) [(p ∨ q) ∧ (∼ q)] ⇒ p (n) [(p ⇒ q) ∧ (r ⇒ s) ∧ (p ∨ r)] ⇒ (q ∨ s) It should be emphasized that finding a few examples is not a formal proof of a universal statement. Consider the statement “for all integers n ≥ 2, f (n) = 2n − 1 is a prime number.” One can observe that f (2) = 3, f (5) = 31, f (7) = 127, etc. are prime numbers. There are, however, many integers n such that f (n) are not primes. What we can claim is, “there exists integers n ≥ 2 such that f (n) = 2n − 1 are prime numbers,” albeit not a very useful result. Table 1.1 summarizes some useful tautologies used in proof. There is one more important tool called mathematical induction. But we defer the discussion until chapter 2, where we explore a little bit more about natural numbers.

1.5

Strategies in Mathematical Proof

Constructing a formal mathematical proof is a creative process. And there is no unique way to achieve that. In the chapters that follow you will read and work on many proofs, some are straight forward and some are tricky. Many times you will encounter a statement that you have no clue where to start. Here are some recommendations to follow and kick start the thinking process. First, decipher the logical structure of the statement. Is it a simple statement, a conditional statement, or a quantified statement? On a side note to your formal proof, write down what are given in the statement, and what are needed to be proven. Second, recall and write down the definitions of every mathematical objects in 14

1.6. EXERCISES the statement. A lot of simple proofs just need a few steps between the definitions. In making those steps, recall any known results that you or others have developed. Sometimes an example or a diagram may help you to clarify some ideas. Keep in mind, however, that examples and diagrams are not formal proofs. One exception is that you are finding a counter-example to disprove a universal statement. For example, to disprove a universal statement such as ∀x ∈ R, p(x), all you need is to find one number x such that p(x) is false. On the other hand, to prove such a statement, always begin your proof with “Let x be a real number. . . . .” My final advice is that studying mathematics is not unlike learning a foreign language or participating in a sport. Practising is as important as reading and memorizing. Therefore working on the exercises at the end of each chapter is an important part of developing intuition, stimulating creativity, and gaining experience. Over time you will be able to acquire that ellusive quality called mathematical maturity. As one Olympic swimmer once said, “To be a good swimmer, you have to feel the water.”

1.6

Exercises

1. Determine whether each of the following sentences is a statement. Provide a brief justification. (a) Some sentences can be labelled true and false. (b) All students who fail the first test should drop the course. (c) If Hillary Clinton becomes the president of the United States in 2016, she will be the best president in U.S. history. (d) The contrapositive of an implication is not equivalent to the implication. (e) When the central bank increases the supply of money, the interest rate will decrease and the quantity of money demanded will increase. 2. Determine whether each of the following sentences is a statement. Provide a brief justification. (a) All consumers have transitive preferences. (b) The inflation rate is negatively related to the unemployment rate. (c) All students should study mathematics. 15

CHAPTER 1. LOGIC AND PROOF (d) This statement is not true. (e) This is not a statement. (f) If f : X → Y and h : X → Z are linear functions with kernel f ⊆ kernel h, then there exists a linear function g : f (X) → Z such that h = g ◦ f . 3. Determine whether each of the following statements is true or false. Provide a brief explanation. (a) All presidents of the United States, past or present, are men. (b) Given the function f (x) = x2 . If f (x1 ) = f (x2 ), then x1 = x2 . (c) If the equilibrium price of a commodity goes up, then the equilibrium quantity must decrease. (d) For all  > 0, there exist δ < 0 such that δ ≥ . (e) This statement is not true. 4. Consider the compound statement: If f is differentiable on an open interval (a, b) and if f assumes its maximum or minimum at a point c ∈ (a, b), then f 0 (c) = 0. (a) Express the statement in four simple statements p, q, r, and s with logical symbols ⇒, ∀, ∃, 3, ∨, ∧, etc. (b) Find the negation of the statement in logical symbols. 5. If limx→a f (x) 6= f (a), then f is not continuous at a. Also, f is differentiable at a implies that f is continuous at a. Prove that f is not differentiable at a whenever limx→a f (x) 6= f (a). 6. Let p and q be logical statements. Prove the following De Morgan’s laws: (a) ∼ (p ∧ q) ⇔ (∼ p) ∨ (∼ q). (b) ∼ (p ∨ q) ⇔ (∼ p) ∧ (∼ q). 7. Determine if an implication and its converse are equivalent. 8. Determine if an implication and its inverse are equivalent. 9. Consider the following statements:2 2

Neil Irwin, “Welcome to the Everything Boom, or Maybe the Everything Bubble,” New York Times, July 7, 2014.

16

1.6. EXERCISES “Around the world, nearly every asset class is expensive by historical standards. . . . The inverse of that is relatively low returns for investors.” Is the second statement the logical inverse of the first statement? Discuss. 10. Prove the following characterizations of an implication by constructing truth tables. (a) (p ⇒ q) ⇔ [(∼ q) ⇒ (∼ p)]. (b) (p ⇒ q) ⇔ [(∼ p) ∨ q]. (c) (p ⇒ q) ⇔ [p ∧ (∼ q) ⇒ c]. 11. Use a truth table to show that each of the following statement is a tautology: (a) [∼ q ∧ (p ⇒ q)] ⇒ ∼ p. (b) ∼ (p ⇒ q) ⇔ (p ∧ ∼ q). 12. A political commentator asserts the statement p ⇒ q where • p: For a number of reasons, country X cannot have democracy. • q: Democratic movement is suppressed by the government. (a) State the converse of the above statement in plain English. (b) State the contrapositive of the above statement in plain English. (c) If the government suddenly changes its view and promotes democracy in country X, what would the commentator say? 13. A prime number is a natural number that is divisible by 1 and itself only. Let P be the set of all prime numbers. Consider the statement “If n is a prime number, then 2n + 1 is also a prime number.” (a) Write the statement in logical and set symbols. (b) Write the contrapositive of the statement in plain English. (c) Prove or disprove the statement. 14. There exists a number x such that for all y, f (x, y) ≥ 0. (a) Write the above statement in logical and set symbols. 17

CHAPTER 1. LOGIC AND PROOF (b) Find the negation of the statement. 15. In each part below, the hypotheses are assumed to be true. Use the tautologies from table 1.1 to establish the conclusion. Indicate which tautology you are using to justify each step. (a) Hypotheses: r, ∼ t, (r ∧ s) ⇒ t Conclusion: ∼ s (b) Hypotheses: s ⇒ p, s ∨ r, q ⇒ ∼ r Conclusion: p ∨ ∼ q (c) Hypotheses: ∼ s ⇒ t, t ⇒ r Conclusion: ∼ r ⇒ s (d) Hypotheses: r ⇒ (∼ p ∨ q) Conclusion: p ⇒ (q ∨ ∼ r) 16. Repeat the above exercise with the following statements. (a) Hypotheses: r ⇒ ∼ p, ∼ r ⇒ q Conclusion: p ⇒ q (b) Hypotheses: q ⇒ ∼ p, r ⇒ s, p ∨ r Conclusion: ∼ q ∨ s (c) Hypotheses: p ⇒ s, s ⇒ q, ∼ q Conclusion: ∼ p (d) Hypotheses: ∼ s ⇒ ∼ p, (∼ q ∧ s) ⇒ r Conclusion: p ⇒ (q ∨ r) 17. Let n be a positive integer. Prove or disprove the following statements: (a) 2n is divisible by 4. (b) If n2 is an odd number, then n is an odd number. 18. Let n be a positive integer. Prove or disprove the following two statements: (a) If n is divisible by 2 and by 5, then it is divisible by 10. (b) If n is divisible by 4 and by 6, then it is divisible by 24. (c) Can you prove the general statement? If n is divisible by l and k, which are distinct prime numbers. Then n is divisible by lk. 18

1.6. EXERCISES 19. Let x be a real number. Prove or disprove the following statement: If |x − 2| ≤ 3, then −1 ≤ x ≤ 5. 20. Let x be a rational number. Prove that if xy is an irrational number, then y is an irrational number. 21. Prove or disprove: If 5x is an irrational number, then x is also an irrational number. 22. Prove or disprove: For every positive integer n, the function given by f (n) = n2 + n + 13 gives a prime number. 23. Prove that if n is an odd integer, then n2 + 2n − 1 is even. 24. Let m and n be integers. Prove that mn is even if and only if m is even or n is even. 25. Let x be a non-zero real number. If x+

1 < 2, x

then x < 0. (a) Prove the statement directly. (b) Repeat using the contrapositive. (c) Repeat using proof by contradiction. 26. Suppose that p is a prime number. Show that

19



p is irrational.

CHAPTER 1. LOGIC AND PROOF

20

Chapter 2 Sets and Relations The language of sets is an important tool in defining mathematical objects. In this chapter we review and study set theory from an intuitive and heuristic approach. Readers who want to get the formal axiomatic treatment can consult books on set theory such as Devlin (1993).

2.1

Basic Notations

Although not a formal definition, a set can be understood to be a collection of welldefined objects. Each individual object is called an element or a member of the set. We express the idea that an object a is a member of a set A by a ∈ A, where the symbol ∈ reads “is an element of”, “belongs to”, “is contained in”, or simply “is in”. Normally, but not always, we use lower case letters to represent the elements and upper case for sets. Instead of ∼ (a ∈ A), which means that a is not an element of A, we write a ∈ / A. There are several ways to describe a set. For examples, A = {Sonata, 7, tree, $2.75, utility, π, C++, entropy}, B = {1, 2, 3, . . . , 9}, C = {x : x is a day of the week}. We describe the set A by listing all of the elements between the braces { }. Elements are separated by commas. Although the elements seem to be unrelated objects but nevertheless their meanings are well-defined. The set B contains the integers from 1 to 9. The elements represented by the ellipsis “. . . ” are to be understood in the context. The set C contains any element x which satisfies the property described 21

CHAPTER 2. SETS AND RELATIONS after the colon (:). Therefore it can also be listed as C = {Sunday, Monday, Tuesday, Wednesday, Thursday, Friday, Saturday}. In general, if p(x) is a statement whose truth value depends on the variable x, then the set {x : p(x)} contains all elements x such that p(x) is true. Being a general idea, a set can contain elements which are themselves sets. For example, S = {A, B, C}, where A, B, and C are sets defined above. A set of sets is often called a class or a collection. Given two sets A and B, we say A is a subset of B, or A ⊆ B, if x ∈ A implies that x ∈ B. That is, all the elements belongs to A are also in B. Two sets are equal if they contain the same elements. In other words, (A = B) ⇔ [(A ⊆ B) ∧ (B ⊆ A)]. The set A is said to be a proper subset of B if A ⊆ B but A 6= B. Therefore, there exists some elements in B that are not in A. In this case we write A ⊂ B. Some important sets in mathematics are listed below: Natural Numbers N = {1, 2, 3, . . . } Integers Z = {. . . , −3, −2, −1, 0, 1, 2, 3, . . . } Rational Numbers Q = {n/m : n, m ∈ Z, m 6= 0} Real Numbers R Complex Numbers C = {a + bi : a, b ∈ R, i =



−1}

The set of natural numbers N and the number zero can be formally constructed with set theory using an idea called ordinal. Alternatively, we can accept the fact that many animals, including humans, have the innate ability to count and to add.1 Then given a natural number n, we ask the question whether there exists a number x such that n + x = 0. This gives rise to the idea of negative integers. Next, given two integers n and m, we ask the question if there exists a number x such that n = mx. The results are the rational numbers. We shall not give a precise definition of the set of real numbers here. Interested readers can find the process of constructing 1

See Holt (2008) or Economist (2008).

22

2.1. BASIC NOTATIONS real numbers from the rational numbers in many books on analysis.2 The set R √ √ contains all rational numbers in Q and all irrational numbers such as 2, 5, π, etc. Geometrically, it contains all points on the real line. You should convince yourself that N ⊂ Z ⊂ Q ⊂ R ⊂ C. In a mathematical system, it is often desirable to define a universal set U which contains all the elements under discussion. For example, in number theory, the universal set is the set of integers U = Z. In real analysis, we often refer to the set of real numbers U = R. By definition, any set under discussion is a subset of U . On the other hand, it is convenient to define a set with no element, which is called the empty set ∅. The empty set is a subset of any set. Given a set S, the power set of S, denoted by P(S), is the collection of all subsets of S. Example 2.1. Let S = {a, b, c}. Then P(S) = {∅, {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c}}. The cardinality of a set S is defined as the number of distinct elements in the set and is denoted by |S|. For example, in the sets defined above, |A| = 8,

|B| = 9,

|C| = 7,

|S| = 3,

|P(S)| = 8.

These are called finite sets because they contain a finite number of elements. Sets whose elements can be listed one by one are called countable or denumerable. For example, N, Z, and Q are countable sets. Also, any finite set is countable. On the other hand, R and C are uncountable. Theorem 2.1. Let S be a finite set with cardinality |S| = n. Then |P(S)| = 2n . Proof. We shall give an informal proof here. In each subset of S, an element is either in the set or not in the set. Therefore there are two choices for each element. The total number of combinations is therefore 2 × 2 × · · · × 2 = 2n for all the n elements of S. 2

See, for example, Rudin (1976). The colourful description of the number systems in Strogatz (2010) is entertaining.

23

CHAPTER 2. SETS AND RELATIONS Since the natural numbers are unbounded, there is an infinite number of them. This is also true for Z, Q, R, and C. It turns out, however, that there are different sizes of infinities. The cardinality of the set of natural numbers is denoted by |N| = ℵ0 (pronounced as aleph naught), while the cardinality of the set of real number is denoted by |R| = c. Some facts on cardinality are listed as follows.3 1. For any set S, |S| < |P(S)|. 2. |N| = |Z| = |Q| = ℵ0 . 3. |P(N)| = c. Therefore ℵ0 = |N| < |R| = c. We should be cautious that infinity, denoted by ∞, is a mathematical symbol of unboundedness. For example, an interval on the real line (a, ∞) = {x ∈ R : x > a} represents the set of all real numbers greater than a. We do not generally treat ∞ as a number. More importantly, infinity is implied by the logical structure of the number systems. As a concept in science, however, it is unreachable, unobservable, and untestable. This distinction between mathematics and reality is emphasized by Ellis (2012) in what he calls Hilbert’s Golden Rule: If infinities occur as a core part of your explanatory model, it is not science. (p. 27) As a consequence, we use infinity in scientific and economic models just to indicate that a variable under consideration is a large number.

2.2

Set Operations

Given two or more sets, we frequently merge, subtract, separate, concentrate, and negate them. In this section we define the formal rules of manipulating sets. Suppose that A and B are subsets of a universal set U . The intersection of A and B, denoted by A ∩ B, is a set containing elements both in A and B. That is, A ∩ B = {x : (x ∈ A) ∧ (x ∈ B)}. 3

For a formal treatment, see for example Gerstein (2012). A brief discussion can be found in Matso (2007).

24

2.2. SET OPERATIONS U

A

U

B

A

(a) A ∩ B

B

(b) A∆B

Figure 2.1: Venn Diagrams The union of A and B, denoted by A ∪ B, is a set containing element either in A or in B, or both: A ∪ B = {x : (x ∈ A) ∨ (x ∈ B)}. It is a trivial exercise to show that (A ∩ B) ⊆ (A ∪ B). Two sets A and B are said to be disjoint if they have no element in common. That is, A ∩ B = ∅. The difference of two sets, denoted by A \ B, is the set which contains the elements in A but not in B: A \ B = {x : (x ∈ A) ∧ (x ∈ / B)}. The set Ac is called the complement of A, which contains all the element not in A. In other words, Ac = U \ A.4 Example 2.2. Prove the first De Morgan’s law of sets: (A ∪ B)c = Ac ∩ B c . Proof. To show equality, we need to show that (A ∪ B)c ⊆ Ac ∩ B c and Ac ∩ B c ⊆ (A ∪ B)c . So first suppose that x ∈ (A ∪ B)c . This means that x ∈ / (A ∪ B), which implies that x ∈ / A and x ∈ / B. By definition of the complement this means c c c x ∈ A ∩ B . Therefore (A ∪ B) ⊆ Ac ∩ B c . Conversely, let x ∈ Ac ∩ B c . Then x is not in A and also not in B, which can be written as x ∈ / A ∪ B. Again by definition x ∈ (A ∪ B)c . Hence Ac ∩ B c ⊆ (A ∪ B)c as required. 4

¯ Some writers denote the complement of A by A0 , while others prefer A.

25

CHAPTER 2. SETS AND RELATIONS Before attempting a formal proof, it is sometimes helpful to illustrate a set operation with a diagram. Often, in a Venn diagram, we represent the universal set with a rectangle, in which all subsets are drawn inside. Figure 2.1(a) depicts the intersection of two sets A and B, while 2.1(b) illustrates their symmetric difference, which is defined as A ∆ B = (A \ B) ∪ (B \ A). Let A1 , A2 , A3 , . . . be a collection of sets. Suppose we want to form the union of all sets in the collection. We can denote the operation as [

An = A1 ∪ A2 ∪ A3 ∪ · · · .

n∈N

Similarly, the intersection of the sets can be denoted by \

An = A1 ∩ A2 ∩ A3 ∩ · · · .

n∈N

These are examples of indexed collection of sets. The set N is called an index set. The index set can be finite, infinitely countable, or uncountable. Example 2.3. Let An = (−1/n, 1/n), n ∈ N. Then [

An = (−1, 1),

n∈N

\

An = {0}.

n∈N

Let S be a set and I an index set. We say that an indexed collection of subsets of S, {Aα ⊆ S : α ∈ I}, is a partition of S if 1. for all α ∈ I, Aα 6= ∅, that is, the subsets are nonempty; 2. for all α, β ∈ I such that Aα 6= Aβ , Aα ∩ Aβ = ∅, that is, distinct subsets are pairwise disjoint; 3. ∪α∈I Aα = S. Example 2.4. Suppose that S = {x : x is a day of the year}, I = {α : α is a day of the week}. 26

2.2. SET OPERATIONS

A Delicious Partition

Then S is partitioned into seven subsets: A1 = {x ∈ S : x is a Monday}, A2 = {x ∈ S : x is a Tuesday}, .. . A7 = {x ∈ S : x is a Sunday}. Example 2.5. The indexed collection {An = (n, n + 1] : n ∈ Z} forms a partition of the set R. In defining a set, the order of the elements in the list is not important. For example, {a, b} = {b, a}. In some situations, however, we want to define an ordered set. For example, an ordered pair of two elements a and b is defined as (a, b) = {{a}, {a, b}}, whereas (b, a) = {{b}, {a, b}}. It is obvious that (a, b) 6= (b, a). Therefore, given two ordered pairs, (a, b) = (c, d) if and only if a = c and b = d. We are now ready to define the Cartesian product of two sets A and B: A × B = {(a, b) : a ∈ A, b ∈ B}. In general, A × B 6= B × A unless A = B. The concept can be expanded into 27

CHAPTER 2. SETS AND RELATIONS multi-dimensional product sets. Let A1 , A2 , . . . , An . Then A1 × A2 × · · · × An = {(a1 , a2 , . . . , an ) : ai ∈ Ai , i = 1, 2, . . . , n}. The elements in the product set, (a1 , a2 , . . . , an ), are called n-tuples. If A1 = A2 = · · · = An = A, we write the product set as An . The followings are examples of product sets. Example 2.6. Let A = {a, b} and B = {1, 2, 3}. Then A × B = {(a, 1), (a, 2), (a, 3), (b, 1), (b, 2), (b, 3)}. Generally, if A and B are finite sets with |A| = n and |B| = m, then |A × B| = nm. Example 2.7. The set of complex numbers C is the product set R2 with two operations, namely addition and multiplication. For any (a, b) and (c, d) in C, (a, b) + (c, d) = (a + c, b + d), (a, b) × (c, d) = (ac − bd, ad + bc). The first number a in (a, b) is call the real part and the second number b is the imaginary part. Example 2.8. Define R+ = {x ∈ R : x ≥ 0}, that is, the set of nonnegative real numbers. In consumer analysis, x = (x1 , x2 , . . . , xn ) ∈ Rn+ is called a consumption bundle with n goods and services. Each component xi in the n-tuple represents the quantity of product i in the bundle. Similarly, define R++ = {x ∈ R : x > 0}, the set of positive real numbers. Then the corresponding prices of the goods and services in a bundle is the n-tuple p = (p1 , p2 , . . . , pn ) ∈ Rn++ . Total expenditure is therefore the dot product of the two n-tuples, defined by p · x = p 1 x1 + p 2 x2 + · · · + p n xn . Example 2.9. In production theory, an n-tuple y = (y1 , y2 , . . . , yn ) ∈ Y ⊂ Rn represents a net output vector of a firm. A negative value for a component yi implies that good or service i is an input factor in the production process, otherwise it is an output good or service. The production technology chosen by the firm determines the shape of the production set Y . The dot product of prices and the net output 28

2.3. BINARY RELATIONS vector is the profit of the firm: π = p · y = p1 y1 + p2 y2 + · · · + pn yn .

2.3

Binary Relations

Given two sets A and B, a relation R between A and B is a subset of A × B. That is, R ⊆ A × B. If A = B, we say that R is a relation on A. Instead of (a, b) ∈ R, we often write a R b, or a is related to b by R. Example 2.10. Let N be a set of nuts and B be a set of bolts. We can define (n, b) ∈ R if the size of nut n fits the size of bolt b. Example 2.11. Let A be the set of people. A relation R on A can be defined as “is the mother of”. The statement “Mary is the mother of James.” can be expressed as (Mary, James) ∈ R. Example 2.12. A relation R on N can be defined as “is a prime factor of”. For examples, (5, 20) and (7, 56) are in R but (3, 13) ∈ / R. Example 2.13. Let m, n ∈ Z. We say m divides n if there exists k ∈ Z such that n = km. Instead of m R n, we have a special symbol for the relation, m|n. For examples, 4|32, 7|49, 12|132 but 3 - 23. Example 2.14. Let Rn+ be the consumption set of n goods and services of a consumer. We can define a preference relation % on Rn+ . For consumption bundles a, b ∈ Rn+ , a % b means that bundle a is at least as good as bundle b. Given a relation R between A and B, we call the set A the domain of R and B the codomain of R. The range of R is defined as the set of elements in the codomain that is in the relation. That is, Range R = {b ∈ B : (a, b) ∈ R for some a ∈ A}. By definition the range is a subset of the codomain of a relation. We can also define an inverse relation R−1 from B to A such that b R−1 a ⇔ a R b, 29

CHAPTER 2. SETS AND RELATIONS or formally, R−1 = {(b, a) ∈ B × A : (a, b) ∈ R}. Notice that the domain of R−1 is B and the codomain is A. There are some properties that we frequently encounter in a binary relation R on a set A. In the following definitions, a, b, and c are arbitrary elements in A. Complete: (a R b) ∨ (b R a). Reflexive: a R a. Symmetric: (a R b) ⇒ (b R a). Transitive: [(a R b) ∧ (b R c)] ⇒ (a R c). Circular: [(a R b) ∧ (b R c)] ⇒ (c R a). Asymmetric: (a R b) ⇒ ∼ (b R a). Antisymmetric: [(a R b) ∧ (b R a)] ⇒ (a = b). Example 2.15. Consider the relation “divides” on Z from example 2.13. As the example shows, not any pair of numbers divides each other so the relation is not complete. It is true that for all n ∈ Z, n|n. Therefore the relation is reflexive. If m|n, it is not necessary that n|m unless m = n. It follows that the relation is not symmetric but antisymmetric. If m|n and n|p, then by definition there exists k, l ∈ Z such that n = km and p = ln. Hence p = (lk)m so that m|p but in general p - m. We conclude that | is transitive but not circular. Finally, reflexivity implies that the relation is not asymmetric. A relation R on a set S is called a partial order if R is reflexive, transitive, and antisymmetric. The set S is called a partial-ordered set, or poset. For example, the relation m|n on Z, as shown in example 2.15, is a partial order. Example 2.16. Let S be a set. The relation ⊆ on the power set of S, P(S), is a partial order. A complete partial order is called a linear order. For example, the relation ≥ on R is a linear order. Notice that completeness implies reflexivity (but the converse is not true) so the latter assumption is redundant. Another important class of relations in mathematics is the equivalence relation, which is defined as a relation R on a set S that is reflexive, transitive, and 30

2.3. BINARY RELATIONS symmetric. The relation “=” on the set of real numbers R is an example of equivalence relation. An equivalence relation is often denoted by an ordered pair (S, ∼), where we use the symbol ∼ in place of R. For any a ∈ S, we define the equivalence class of a as ∼ (a) = {x ∈ S : x ∼ a}. Example 2.17. In example 2.7 we express a complex number as an ordered pair in R2 . For any complex number z = (a, b), we define its absolute value as |z| = √ a2 + b2 . Then the relation z ∼ w, which means that |z| = |w|, is an equivalence relation. The equivalence class ∼ (z) contains all the numbers that have absolute values equal to |z|. For example, the conjugate of z, z¯ = (a, −b), belongs to ∼ (z). The following theorem shows that the collection of the distinct equivalence classes in fact forms a partition of the set. Theorem 2.2. Suppose that (S, ∼) is an equivalence relation where S is nonempty. Then the collection of equivalence classes {∼ (a) : a ∈ S} is a partition of S. Proof. By reflexivity, for every a, b ∈ S, ∼ (a) and ∼ (b) are nonempty. Suppose that ∼ (a) ∩ ∼ (b) 6= ∅. Then there exists an x ∈ ∼ (a) such that x is also in ∼ (b). This means that a ∼ x and x ∼ b, and by transitivity, a ∼ b. It follows that ∼ (a) = ∼ (b). Finally, a ∈ ∼ (a) for all a ∈ S. Therefore ∪a∈S ∼ (a) = S. We conclude that {∼ (a) : a ∈ S} is a partition of S. We can also show that the converse of theorem 2.2 is also true. Theorem 2.3. Suppose that I is an index set and {Aα ⊆ S : α ∈ I}, is a partition of S. Then there exists an equivalence relation (S, ∼) such that for each a ∈ S, the equivalence class ∼ (a) = Aα for some α ∈ I. Proof. Define a relation R on S such that, R = {(a, b) ∈ S 2 : a, b ∈ Aα for some α ∈ I}. Obviously for all a ∈ S, a R a so that R is reflexive. For any (a, b) ∈ R, we have (b, a) ∈ R by definition. Therefore R is symmetric. Suppose that (a, b) and (b, c) are in R. Then a, b ∈ Aα for some α ∈ I. Also, b, c ∈ Aβ for some β ∈ I. Since b is in both Aα and Aβ , Aα = Aβ . Therefore (a, c) ∈ R so that R is transitive. Therefore, a partition on a set S induces an equivalence relation on S and vice versa. They are the two sides of the same coin. 31

CHAPTER 2. SETS AND RELATIONS Example 2.18. In example 2.4 we partition the days in a year into seven subsets, A1 , . . . , A7 , according to the day of the week. We can define an equivalence relation as a ∼ b if a and b belong to the same day of the week. If a is a Monday, then ∼ (a) = A1 . If b is a Tuesday, then ∼ (b) = A2 , and so on. In the partial ordering and equivalence relations we require them to be antisymmetric and symmetric respectively. A more general relation is a preorder, which is defined as reflexive and transitive. It is sometimes called a preference relation or simply an ordering and is denoted by (S, %). In what follows we shall call it an ordering. If a % b in an ordering, we say that a succeeds b, or, alternatively, b precedes a. Example 2.19. In example 2.14 we define a consumer preference relation % on the consumption set Rn+ . In consumer theory we generally assume that the relation is complete (and hence reflexive) and transitive. Therefore mathematically % is an ordering. With this basic preference relation we also define two more relations. Suppose that a, b ∈ Rn+ : 1. If a % b and b % a, then we say bundle a is indifferent to bundle b, which is denoted by a ∼ b. In fact we impose symmetry on the ordering and turn it into an equivalence relation. The equivalence class of a bundle, ∼ (a), is called the indifference set of a in economics. 2. If a % b and b  a, then we say bundle a is strictly preferred to bundle b, which is denoted by a  b. Mathematically, we drop reflexivity and add asymmetry to the ordering and turn it into a strict ordering. With the above notations on an ordering (S, %), we can define different types of intervals: [a, b] = {x ∈ S : (x % a) ∧ (b % x)}, (a, b) = {x ∈ S : (x  a) ∧ (b  x)}, (a, b] = {x ∈ S : (x  a) ∧ (b % x)}, [a, b) = {x ∈ S : (x % a) ∧ (b  x)}. The first definition is called a closed interval, the second an open interval. The last two definitions are sometimes called half-open intervals. Unbounded intervals are expressed as contour sets. In particular, for any a ∈ S, the upper contour set 32

2.3. BINARY RELATIONS of a is defined as % (a) = {x ∈ S : x % a}, whereas the lower contour set of a is - (a) = {x ∈ S : a % x}. You should convince yourself that % (a) ∩ - (a) = ∼ (a). An element m of an ordering (S, %) is called a maximal element if for all x ∈ S, x  m. That is, there is no element in S strictly succeeding m. An alternative definition is that for all x ∈ S, if x % m, then x ∼ m. An element g is called the greatest element if g % x for all x ∈ S, that is, g succeeds all elements in S. Notice that in the definition of an maximal element, we do not assume that the ordering is complete. Therefore there can be elements in S which are not related to m. In the definition of the greatest element, however, g succeeds all other elements in S so that it is assumed to relate to all elements in S. The greatest element of a set A must be a maximal element, but the converse is not true. The definitions of a minimal element and the least element are left as an exercise. Example 2.20. The countries which constitute the United Nations are supposed to be equal. Therefore the head of states of the countries are unrelated to each other in their lines of authority so that all of them are maximal elements at the U.N. Within its own government, however, each head of state is the greatest element in political ranking. Example 2.21. In consumer theory we usually assume that the consumption set Rn+ has no greatest element under the preference relation % since the set Rn+ is unbounded. Nevertheless, the choice set of the consumer is always a bounded subset of Rn+ because of financial and other constraints. A maximal element in the choice set is called an optimal bundle. Let A be a subset of an ordered set S. An element s ∈ S is an upper bound of A if for all x ∈ A, s % x. If an upper bound exists we call A bounded above. Let B be the set of all upper bounds of A, that is, B = {s ∈ S : s % x, x ∈ A}. Then the least element or a minimal element in B is called the least upper bound or supremum of A, which is denoted by sup A. Notice that sup A may or may not 33

CHAPTER 2. SETS AND RELATIONS be an element of A. If it does, sup A is also a maximal element of A, often denoted by max A. Similarly, An element e ∈ S is a lower bound of A if for all x ∈ A, x % e. If a lower bound exists we call A bounded below. Let L be the set of all lower bounds of A. Then the greatest element or a maximal element in L is called the greatest lower bound or infimum of A, which is denoted by inf A. Again inf A may or may not be an element of A. If it does, it is also a minimal element of A, often denoted by min A. The set A is said to be bounded is it is bounded above and bounded below. Theorem 2.4. Let S be a linear order and suppose that A is a nonempty bounded subset of S. Then sup A is unique. Proof. Suppose that x and y are both sup A. Then by definition x % y and y % x. Since a linear order is antisymmetric, x = y. The uniqueness of inf A of a linear order is left as an exercise. Consequently, every nonempty bounded subset of R has a unique supremum and a unique infimum. Example 2.22. Let A = {1/n : n ∈ N} ⊆ R. Then sup A = 1 and inf A = 0. Notice that inf A does not belongs to A.

2.4

Well-Ordering and Mathematical Induction

A relation R on a set S is a well-ordering if it is 1. complete, 2. transitive, 3. antisymmetric, 4. every nonempty subset of S has a least element. That is, S is a linearly ordered set with a least element for every subset. The sets Z, Q, and R are not well-ordered with respect to the linear order ≥. For example, the set of negative integers, {. . . , −3, −2, −1}, does not have a least element. The set {1/n : n ∈ N}, either as a subset of Q or of R, do not have a least element as well. It turns out that the set of natural numbers N satisfies the requirements. We generally regard this property as an axiom called the Well-Ordering Principle: 34

2.4. WELL-ORDERING AND MATHEMATICAL INDUCTION Axiom 2.1. If A is a nonempty subset of N, then there exists a number l ∈ A such that for all k ∈ A, k ≥ l. With the well-order principle, we can prove a useful tool in proof called the Principle of Mathematical Induction. It involves proving statements that depend on the set of natural numbers, which economists frequently encounter in dynamic models. Theorem 2.5. Let p(n) be a statement whose truth value depends on n ∈ N. Suppose that 1. p(1) is true, 2. for all k ∈ N, p(k) implies p(k + 1). Then p(n) is true for all n ∈ N. Proof. Suppose that p(1) is true and p(k) implies p(k + 1) for any integer k > 1. Suppose on the contrary that there exists m such that p(m) is false. Let the nonempty set A = {m ∈ N : p(m) is false}. By axiom 2.1, there exists a number l ∈ A such that m ≥ l for all m ∈ A. Since p(1) is true, l must be greater than 1. Then l − 1 ∈ / A since l is the least element in A. This means that p(l − 1) is true. By hypothesis p(l) is true, contradicting the fact that l ∈ A. We conclude that p(n) is true for all n ∈ N. To apply the principle of mathematical induction to prove that a statement p(n) is true for n = 1, 2, 3, . . . , we need to show that p(1) is true and for any k ∈ N, p(k) implies p(k + 1). Example 2.23. Show that for all n ∈ N, n X

i=

i=1

n(n + 1) . 2

P Proof. For n = 1, ni=1 i = n(n + 1)/2 = 1 so that p(1) is true. Next, assume that p(k) is true for any k, that is, k X i=1

i=

k(k + 1) . 2 35

CHAPTER 2. SETS AND RELATIONS Then k+1 X i=1

i =

k X

i + (k + 1)

i=1

k(k + 1) = + (k + 1) 2 (k + 1)(k + 2) = . 2 This shows that p(k + 1) is true and so by induction p(n) is true for all n ∈ N. For any number m ∈ N, the set S = {n ∈ N : n ≥ m} is a well-ordered set. The principle of mathematical induction can be modified to prove statements such as “for all n ∈ S, p(n) is true”. In this case, we need to show that p(m) is true and for all k > m, p(k) implies p(k + 1). Example 2.24. Show that 2n ≥ n2 for all integers n ≥ 4. Proof. For n = 4, 24 = 16 = 42 and so p(4) is true. Now assume that 2k ≥ k 2 for any k > 4. Then 2k+1 = 2(2k ) ≥ 2(k 2 ) ≥ k 2 + 5k

(since k > 4)

= k 2 + 2k + 3k ≥ k 2 + 2k + 1 = (k + 1)2 . Therefore by induction p(n) is true for all k ≥ 4.

2.5

Functions

Let A and B be nonempty sets. A function f from A to B, denoted by f : A → B, is a relation such that each element in A is related to at most one element in B. Instead of (a, b) or a R b, we write f (a) = b. Therefore, if f (a) = b and f (a) = c, then b = c in B. A function is often called a mapping or a transformation. As defined in a relation before, the set A is the domain of f , and B is the codomain. 36

2.5. FUNCTIONS For any C ⊆ A, the image of C, denoted by f (C), is a subset of B such that f (C) = {b ∈ B : f (a) = b, a ∈ C}. The image of the domain of f is called the range of f , that is, range f = f (A). It is clear that the range of a function is a subset of the codomain. For any D ⊆ B, the pre-image of D, denoted by f −1 (D), is a subset in A such that f −1 (D) = {a ∈ A : f (a) ∈ D}. For any b ∈ B, we often call the pre-image f −1 (b) the contour of b. Sometimes we want to express the function as a subset of the product set A × B. In this case we call it the graph of f : graph f = {(a, b) ∈ A × B : f (a) = b}. Example 2.25. Let A = {1, 2, 3, 4} and B = {a, b, c}. Define f : A → B by its graph as graph f = {(1, a), (2, c), (4, c)}. Let C = {1, 3} and D = {a, c}. Then 1. domain f = {1, 2, 3, 4}, codomain f = {a, b, c}, range f = {a, c}, 2. f (C) = {a}, f −1 (D) = {1, 2, 4}, 3. the contour of c is f −1 (c) = {2, 4}, and f −1 (b) = ∅. Some special types of functions are defined below. We suppose a function f : A → B: Everywhere defined: Each of the element in the domain is related to an element in the codomain. That is, for all a ∈ A, f (a) = b for some b ∈ B. Injective: Each element in the codomain has at most one pre-image. That is, for all a, c ∈ A, if f (a) = f (c), then a = c. Injective functions are often called one-to-one functions. Surjective: For each b ∈ B, there exists a ∈ A such that f (a) = b. This implies that the range of f is equal to the codomain. Surjective functions are often called onto functions 37

CHAPTER 2. SETS AND RELATIONS Bijective: Everywhere defined, injective, and surjective. Bijective functions are also called one-to-one correspondences or bijections. Example 2.26. The function defined in example 2.25 is not everywhere defined since the element 3 has no image. It is not injective because 2 and 4 have the same image. Also, it is not surjective since b has no pre-image. Example 2.27. The function f : R → R defined by f (x) = 1/x is not everywhere defined because f (0) is undefined. We can, however, let A = R\{0}. Then f : A → R is everywhere defined. The new definition is also a one-to-one function. Is it onto? Example 2.28. The rotation function f : R2 → R2 is given by f (x1 , x2 ) = (x1 cos θ − x2 sin θ, x1 sin θ + x2 cos θ), where θ ∈ [0, 2π). It rotates a point in the anticlockwise direction by the angle θ. It can be shown that the function is a bijection. Suppose that f : A → B is a bijection. Since f is one-to-one, the pre-image f (b) for every b ∈ B has at most one element in A. Therefore f −1 : B → A is a function from B to A. Since f is onto, f −1 is everywhere defined. Also, f is everywhere defined means that f −1 is onto. Finally, since f is a function, f −1 is one-to-one. We conclude that f −1 is a bijection from B to A. We call f −1 the inverse function of f . For every a ∈ A and every b ∈ B, −1

f −1 (f (a)) = a,

f (f −1 (b)) = b.

Let f : A → B and g : B → C be functions. Notice that B is the codomain of f and the domain of g. We can define a composite function from A to C and denoted by g ◦ f : A → C as (g ◦ f )(a) = g(f (a)), for any a ∈ A. When the domain and the codomain of a function are the same set, we sometimes call the function an operator on a set. Suppose that f is an operator on A. An element p ∈ A is called a fixed point of f if f (p) = p. Fixed points are important in economic analysis because they represents the equilibria or the steady states of dynamical systems. 38

2.5. FUNCTIONS Example 2.29. Let A = R \ {0} and let f : A → A be defined as f (x) = 1/x. Then f is a bijection. The inverse function is f −1 (y) = 1/y. The points p = −1, 1 are fixed points of f and f −1 . Now let g : A → R be defined as g(x) = x2 . Then g ◦ f : A → R becomes (g ◦ f )(x) = (1/x)2 = 1/x2 . Example 2.30. Let A = R \ {1} and B = R \ {2}. Show that f : A → B defined by 2x + 3 f (x) = x−1 is a bijection. Proof. By inspection f is well-defined in R except for the point x = 1. Therefore it is everywhere defined. Let a, b ∈ A and 2b + 3 2a + 3 = . a−1 b−1 Then (2a + 3)(b − 1) = (2b + 3)(a − 1), which can be reduced to a = b. Therefore f is one-to-one. To show that f is onto, let y ∈ B and consider x = (y + 3)/(y − 2). Then 2(y + 3) + 3y − 6 2(y + 3)/(y − 2) + 3 = = y. f (x) = (y + 3)/(y − 2) − 1 y+3−y+2 Notice that f −1 (y) = (y + 3)/(y − 2). Example 2.31. The following are some special functions that we frequently encounter. Here we let f : A → B be a function. 1. Let c ∈ B. If for all x ∈ A, f (x) = c, then we call f a constant function. 2. Let A = B. If for all x ∈ A, f (x) = x, then f is the identity function. 3. Let B = {0, 1} and S ⊆ A, a characteristic function of S is defined as ( f (x) =

1 if x ∈ S, 0 if x ∈ / S.

In econometric analysis, f (x) is often called a dummy variable. 39

CHAPTER 2. SETS AND RELATIONS 4. The function f is called additive if for all x, y ∈ A, f (x + y) = f (x) + f (y). 5. Suppose that the scalar multiplication αx is defined for all α > 0 and x ∈ A. Then f is called linearly homogenous if f (αx) = αf (x). An important class of functions called functionals or real-valued functions is defined as the case that the codomain is R. Since R is a linear order, it possesses the trichotomy property that for any x, y ∈ R, either x < y, x = y, or x > y. As a result, it induces an ordering on the domain. In particular, let f : S → R. For any a, b ∈ S, we can define an ordering % on S such that (a % b) ⇔ (f (a) ≥ f (b)). Contour sets can be defined using numbers instead of an element in S. For example, the upper contour set %f (x) = {a ∈ S : f (a) ≥ x} consists of all the elements in S that have values greater than or equal to the number x. The subscript f means that the ordering % is induced from f . Similarly the lower contour set of x is -f (x) = {a ∈ S : f (a) ≤ x}. Example 2.32. One of the most important application of functionals in economics is the utility function. A consumer’s preference relation on the consumption set Rn+ is represented by a real-valued utility function U : Rn+ → R. The function U is assumed to be everywhere defined so that the preference relation is complete. For consumption bundles a, b ∈ Rn+ , we say that a is at least as good as b, or a % b if U (a) ≥ U (b). Construction of the utility function from a well-defined preference structure can be found in most textbooks in microeconomics. See, for example, Jehle and Reny (2011, 13–16). Sometimes we want to consider the subset of the product set S × R which is above or below the graph of a functional. The geometry of the set can be used to characterize the function. The epigraph of a functional f : S → R is the subset 40

2.5. FUNCTIONS y f (x) epi f (x)

x1

x2

x

Figure 2.2: Epigraph of a Functional

above the graph and is defined as epi f = {(a, x) ∈ S × R : x ≥ f (a)}. Figure 2.2 depicts the epigraph of the function f on the interval (x1 , x2 ). On the other hand, the hypograph is the subset below the graph: hypo f = {(a, x) ∈ S × R : x ≤ f (a)}. It is clear that (epi f ) ∩ (hypo f ) = graph f . A more general idea of order-preserving functions is the class of monotone functions. Let (A, %A ) and (B, %B ) be ordered sets. A function f : A → B is called an increasing function if for all a, b ∈ A, (a %A b) ⇒ (f (a) %B f (b)). On the other hand, f is called a decreasing function if (a %A b) ⇒ (f (b) %B f (a)). 41

CHAPTER 2. SETS AND RELATIONS

2.6

Exercises

1. Define the following sets with mathematical symbols: (a) The set of odd numbers. (b) The set of even numbers. (c) The set of all positive multiples of seven. 2. Determine which of the followings are sets. Justify your answers. (a) {x : x is a graduate student at your university.} (b) {p : p is a logical statement.} (c) {x ∈ R : x2 + 1 = 0.} (d) {x : x is a right glove.} (e) {x : x is a left chopstick.} 3. Let M denote Mark Messier, E denote the Edmonton Oiler hockey team in 1988, and H denote the National Hockey League. What are the relationships between E, H, and M ? 4. Let the sets A and B be defined as A = {x ∈ R : x ≥ 0}, B = {x = y 2 : y ∈ R}. Show that A = B. 5. Prove the following statements: (a) For any set S, ∅ ⊆ S. (b) The empty set ∅ is unique. 6. Determine the cardinalities of the following sets. (a) A = ∅ (b) B = {∅} (c) C = {∅, {∅}} (d) D = {∅, {∅}, {∅, {∅}}} 42

2.6. EXERCISES 7. Let A = {a, b, c, d, e}, B = {1, 2, 3, 4, 5}, C = {x : x is an alphabet in the word ‘concerto’}, P = {x ∈ N : x is a prime number}. Find the sets A ∪ B, A ∩ C, B ∩ P, A ∩ B, C ∪ B, A \ C, C \ A, B \ P . 8. Let A and B be subsets of a universal set U . Suppose that Ac ∪ B = U . Prove that A ⊆ B. 9. Let A, B, and C be subsets of a universal set U . Show that (a) A \ (B ∪ C) ⊆ (A \ B) ∩ (A \ C), (b) A∆B = (A ∪ B) \ (A ∩ B), (c) A ∪ (U \ A) = U . 10. Let A, B, and C be subsets of a universal set U . Show that (a) (A ∩ B) ⊆ (A ∪ B), (b) A \ B = A ∩ B c , (c) A ∪ (B ∩ C) = (A ∪ B) ∩ (A ∪ C). 11. Prove the second De Morgan’s law of sets: (A ∩ B)c = Ac ∪ B c . 12. Let An = (−1/(1 + n), n], n ∈ N. Find ∩n∈N An and ∪n∈N An . 13. Let A = {1, 2}. Find A2 . What is |P(A2 )|? 14. Let A, B, C, D be sets. Suppose that A ⊆ C and B ⊆ D. Show that A × B ⊆ C × D. 15. Let A and B be sets. Prove that A × B = B × A if and only if A = B, or at least one of A or B is empty. 16. Let (a, b) ∈ R2+ . Define A = {x ∈ R : x ≥ a} and B = {y ∈ R : y ≥ b}. Identify A × B on a diagram. (In consumer theory, A × B is called the “no-worse-than set of the bundle (a, b).) 43

CHAPTER 2. SETS AND RELATIONS 17. Let (a, b) ∈ R2+ . Define C = {x ∈ R+ : x ≤ a} and D = {y ∈ R+ : y ≤ b}. Identify C × D on a diagram. (In consumer theory, C × D is called the “nobetter-than set of the bundle (a, b).) 18. Determine if each of the following relation R on a set S is complete, reflexive, transitive, circular, symmetric, asymmetric, or antisymmetric. Justify your answers. (a) S = {a, b, c}, R = {(a, a), (b, b), (c, c), (c, a), (b, c), (b, a)}. (b) S = {rock, scissors, paper}, a R b means “a beats b”. (c) S is a set of people, a R b means “a is connected to b on Facebook”. (d) S is the set of all commercial and investment banks in an economy, a R b means “Bank a has an interbank loan from Bank b”. 19. Let S be the set of all straight lines in the plane. Define the relation R on S as “is perpendicular to.” Determine if the relation is (a) complete, (b) reflexive, (c) transitive, (d) circular, (e) symmetric, (f) equivalence. Justify your answers. 20. Let S be the set of all statements. Define a relation “⇒” on S which means “implies”. Explain whether the relation is (a) complete, (b) reflexive, (c) symmetric, (d) transitive, (e) anti-symmetric, (f) asymmetric. 44

2.6. EXERCISES 21. Let S be a set. Show that the relation ⊆ on the power set of S, P(S), is a partial order. 22. Determine if the relation R on N defined as “is a prime factor of” a partial order or not. 23. Suppose that a relation R on a set A is reflexive and circular. Show that it is an equivalence relation. 24. Suppose that a relation on R is defined by R = {(x, y) ∈ R × R : x − y ∈ Z}. That is, x − y is an integer. Prove that R is an equivalence relation. 25. Define the relation R = {(x, y) ∈ R2 : |x − y| ≤ 1}. Determine if R is an equivalence relation. 26. Define the relation R = {(m, n) ∈ Z2 : 3|(m − n)}. Determine if R is an equivalence relation. 27. Suppose that (S, ∼) is an equivalence relation. Prove that for all a, b ∈ S, a ∼ b if and only if ∼ (a) = ∼ (b). 28. Find the equivalence relation corresponding to the partition defined in example 2.5. 29. Give the formal definitions of a minimal element and the least element of an ordering (S, %). 30. Consider the ordered set (R, ≥). Let the set A = {−1} ∪ (0, 1). (a) Find the set of upper bounds of A. (b) Find the set of lower bounds of A. (c) Find sup A and inf A. (d) Does A have a minimal element and a least element? 45

CHAPTER 2. SETS AND RELATIONS 31. Let (S, %) be an ordering. For all x and y in S, define the following induced relations as (a) x ∼ y ⇔ x % y and y % x, (b) x  y ⇔ x % y and x  y. Show that if x ∼ y and y  z, then x  z. 32. Let (S, %) be a complete ordering. Prove that for every y ∈ S, (a) % (y) ∪ ≺ (y) = S, (b) % (y) ∩ ≺ (y) = ∅. 33. Let the set S = {1, 2, 3, 4, 5, 6}. Define an ordering (S, %) as 5 % 3  1 % 4 ∼ 2 % 6. Find the following: (a) the interval [4, 3), (b) the upper contour set % (1), (c) the greatest element of S, (d) inf S. 34. Let (S, %) be an ordering. Prove that for every x, y ∈ S, if x % y, then - (y) ⊆ - (x). 35. Let (S, ≥) be a linear order and suppose that A is a nonempty bounded subset of S. Show that inf A is unique. 36. Suppose that % is a preference relation on the consumption set Rn+ . Let a ∈ Rn+ . Is the indifference set ∼ (a) a bounded set? If yes find sup ∼ (a) and inf ∼ (a). 37. Show that for all n ∈ N, 3|(4n − 1). 38. Show that for all n ≥ 4, n2 ≤ n!. (Recall that n factorial is defined as n! = n(n − 1) · · · (2)(1).) 39. Define the function f : R → R by y = x3 . (a) Is f everywhere defined, one-to-one, and onto? Explain. 46

2.6. EXERCISES (b) Find the set of all fixed points of f . (c) Define the hypograph of f . 40. Define a function f : X → R as  1 if x ∈ A, f (x) = 0 if x ∈ / A, where A ⊆ X. (a) Is f one-to-one? (b) Is f onto? (c) What is the graph of f ? (d) What is the hypograph of f ? 41. Determine whether the following functions f : R → R are everywhere defined, one-to-one, and onto. (a) f (x) = sin x, (b) f (x) = tan x, (c) f (x) = x2 . 42. Determine whether the following functions on R are everywhere defined, one-toone, and onto. (a) f (x) = ex , (b) g(x) = log x, (c) h(x) = (g ◦ f )(x). 43. Let A = R \ {2} and B = R \ {5}. Show that f : A → B defined by f (x) =

5x + 1 x−2

is a bijection. 44. Let f : R → R and g : R → R be given by f (x) = cos x and g(x) = x2 + 1. Find the following: 47

CHAPTER 2. SETS AND RELATIONS (a) g([−1, 1]); (b) (g ◦ f )([0, π]); √ (c) (f ◦ g)([0, 2π]); (d) (g ◦ f −1 )([−π, π]). 45. Suppose that f is a function which maps a set X into a set Y . (a) Show that for any B ⊆ Y , f (f −1 (B)) ⊆ B. (b) Under what condition that f (f −1 (B)) = B? (c) Prove your answer in Part (b). 46. Suppose that A, B, and C are sets. Let %A , %B , and %C be ordered relations on A, B, and C respectively. (a) Define a decreasing function f : A → B. (b) Suppose that g : B → C is also a decreasing function. Prove or disprove: the composite function g ◦f :A→C is increasing. 47. Suppose f and g are increasing functions on R. Show that (a) −f is decreasing, (b) f + g is increasing, (c) g ◦ f is increasing. 48. Let f and g be functionals on an ordered set (S, %). Suppose f is increasing and g is decreasing. Prove that f − g is increasing. 49. Let U : Rn+ → R be a utility function. Let g : R → R be a strictly increasing function. Show that g ◦ U is also a utility function representing the preference order %.

48

Chapter 3 Basic Topology 3.1 3.1.1

Introduction to Metric Space Topological Space

Let S be a set. A topology on S is a collection T of subsets of S, called open sets, which satisfies the following: 1. ∅ and S are in T , 2. Any unions of sets in T are in T , 3. Any finite intersections of sets in T are in T . For any set S, the collection T = {∅, S} is a trivial topology. On the other hand, T = P(S), the power set of S, is called the discrete topology. In economics we often focus on a specific type of topological space called metric space.

3.1.2

Metric Space

A metric space consists of a set X, with elements called points, and a metric or distance function ρ : X 2 → R, which maps a pair of of points in X to a real number. The metric satisfies the following axioms: For all x, y, z ∈ X, 1. ρ(x, y) ≥ 0, ρ(x, x) = 0, 2. ρ(x, y) = ρ(y, x), 3. ρ(x, y) + ρ(y, z) ≥ ρ(x, z). 49

CHAPTER 3. BASIC TOPOLOGY The first axiom requires that the metric or distance between two points must be positive, unless x = y. The second axiom is about symmetry. It does not matter if we measure the distance from x to y or from y to x. The last one is called triangular inequality. It is the generalization of the fact that the sum of two sides of a triangle is greater than or equal to the third side. We often denote a metric space by (X, ρ). Any subset of X with the metric ρ also satisfies the above definition and is therefore a metric space itself. Example 3.1. The most important example of metric spaces is the Euclidean space Rn with the metric  1/2 ρ(x, y) = (x1 − y1 )2 + · · · + (xn − yn )2 ,

(3.1)

where x = (x1 , x2 , . . . , xn ) and y = (y1 , y2 , . . . , yn ) are n-tuples in Rn . Notice that for real numbers (n = 1) and complex numbers (n = 2), the Euclidean metric is simply the absolute value of the difference between the two points, that is, ρ(x, y) = |x − y|. For n ≥ 3, the absolute value of an n-tuple x is called the norm of x, defined by kxk = (x21 + x22 + · · · + x2n )1/2 . Therefore the metric defined in (3.1) can be written as ρ(x, y) = kx − yk.

3.1.3

Definitions in Metric Space

In the following definitions let X be a metric space with a metric ρ. Also, A and B are subsets of X. Diameter: The diameter of A is sup{ρ(x, y) : x, y ∈ A}. Bounded Set: A ⊆ X is bounded if its diameter is finite. Open Ball: An open ball of a point p is a set Br (p) = {x ∈ X : ρ(x, p) < r}. The positive number r is called the radius of the ball. Sometimes an open ball is called a neighbourhood of p. Limit Point: A point p is called a limit point of A if every open ball Br (p) contains a point q ∈ A but q 6= p. 50

3.1. INTRODUCTION TO METRIC SPACE Interior Point: A point p is an interior point of A if there exists an open ball of p such that Br (p) ⊆ A. Boundary Point: A point p is a boundary point of A if every open ball of p contains a point in A and a point not in A. The set of all boundary points of A is called the boundary of A, b(A). Open Set: A ⊆ X is open if every point of A is an interior point. Closed Set: A ⊆ X is closed if every limit point of A is a point of A. Closure: The closure of any set A is A¯ = A ∪ b(A). Perfect Set: A closed set A is called perfect if every point in A is a limit point of A. Dense Set: A is dense in X if for all x ∈ X, x is a limit point of A or x ∈ A. ¯ and B ∩ A¯ are empty. Separated Sets: A and B are separated if both A ∩ B Connected Set: A is said to be connected if it is not a union of two nonempty separated sets. Compact Set: A compact set in the Euclidean space can be defined as a closed and bounded set. Convex Set: A subset A in the Euclidean space Rn is convex if for all x, y ∈ A, αx + (1 − α)y ∈ A for 0 < α < 1.

3.1.4

Some Examples

1. Open sets and closed sets are not exclusive. X and ∅ are both closed and open. 2. All open balls are open and convex sets. 3. The limit points of a set A do not necessary belong to A. For example, let A = {1/n : n = 1, 2, . . .}. The point 0 is not in A but it is a limit point of A. ¯ Note that A is neither open nor closed. What is A? 4. The closure of a set A can also be defined as the union of A and all its limit points. 51

CHAPTER 3. BASIC TOPOLOGY ¯ 5. A is closed if and only if A = A. 6. Separated sets are disjoint, but disjoint sets are not necessary separated. For example, the intervals [a, b] and (b, c) are disjoint but not separated. What about (a, b) and (b, c)? 7. The set of rational numbers Q is neither a close set nor an open set in R. It is, however, dense in R. Is it connected? 8. Let X = Rn+ . The budget set B = {x : p · x ≤ M } is compact and convex.1 What is b(B)? 9. A finite set has no limit points and is always compact. 10. An open interval (a, b) is open in R but not in R2 . 11. The sets A = [3, π] and B = {−4, e, π} are both closed in R. But A is perfect while B is not. The proofs of some of the above statements can be found in Rudin (1976).

3.1.5

Sequences

A sequence is a list of points, x1 , x2 , . . . , in a metric space (X, ρ). If the list is finite, it is similar to an n-tuple. Often we use the notation {xn } to represent an infinite sequence. Formally, an infinite sequence is a function from N into X, with f (n) = xn . Notice that not all points in a sequence need to be distinct. The set of all the points in a sequence is called the range of {xn }. The sequence is said to be bounded if the range has a finite diameter. A sequence {xn } in a metric space (X, ρ) is said to converge to a limit x ∈ X if for all  > 0, there exists an integer N such that for all n > N , ρ(xn , x) < . Often a converging sequence is denoted by xn → x or

lim xn = x.

n→∞

A sequence that does not converge is said to diverge. 1

The dot product p · x is defined as p1 x1 + p2 x2 + · · · + pn xn , which will be discussed in the chapter on linear algebra. The inequality means that total expenditure of a consumer on goods and services cannot exceed income M .

52

3.1. INTRODUCTION TO METRIC SPACE Example 3.2. The sequence {xn } in R where xn = 1/n converges to 0. Proof. Let  > 0 be any positive real number. Let N = d1/e. Then for all n > N , ρ(xn , x) = |xn − 0| = 1/n < 1/N ≤ . Therefore xn → 0. Notice that the range of the sequence is infinite, and the sequence is bounded. The following result shows that the concept of the limit of a sequence and a limit point of a set are related. Theorem 3.1. Suppose that A is a subset of a metric space (X, ρ) and p is a limit point of A. Then there exists a sequence in A that converges to p. Proof. Since p is a limit point of A, for any n ∈ N, there is a xn ∈ A such that ρ(xn , p) < 1/n. Now for any  > 0, pick N such that 1/N < . Then for all n ≥ N , we have ρ(xn , p) < 1/n < 1/N < . Therefore xn → p. A Cauchy sequence {xn } in a metric space (X, ρ) is that for all  > 0, there exists an integer N such that ρ(xn , xm ) <  if n ≥ N and m ≥ N . It can be shown that any convergent sequence is a Cauchy sequence. If the converse is also true, (X, ρ) is called a complete metric space. Euclidean spaces are complete. Complete metric spaces have a number of interesting properties and are very useful in economic analysis. Interested readers can consult Chapter 10 in Royden and Fitzpatrick (2010) for more details. A sequence {xn } of real numbers is said to be 1. increasing if xn ≤ xn+1 for all n; 2. decreasing if xn ≥ xn+1 for all n. Increasing and deceasing sequences are called monotone sequences. An useful fact about a monotone sequence is that it converges if and only if it is bounded. The concept of convergence is important in statistics. ¯ n = (X1 + X2 + · · · + Xn )/n denote Example 3.3 (Law of Large Number). Let X the mean of a random sample of size n from a distribution that has mean µ. Then 53

CHAPTER 3. BASIC TOPOLOGY ¯ n } is a sequence in R. The Law of Large Number states that X ¯ n converges to µ {X in probability, that is, for any  > 0, ¯ n − µ| < ) = 1. lim Pr(|X

n→∞

3.1.6

Series in R

Suppose that {an } is a sequence in R. The partial sum of {an } is a sequence {sn } defined by n X s n = a1 + a2 + · · · + an = ai . i=1

We also call the sequence {sn } the infinite series or simply series, which is denoted P by ∞ n=1 an . We say that the series converges to a number s ∈ R if ∞ X

an = s.

n=1

Otherwise we say that the series diverges. Since R is a complete metric space, we can P apply the Cauchy criterion for convergence on the series. That is, ∞ n=1 an converges if an only if for all  > 0, there exists an integer N such that for m ≥ n > N , m X ai < . (3.2) i=n

When n = m, the inequality in (3.2) implies that an → 0. Example 3.4. A geometric series is defined as 2

3

a + ax + ax + ax + · · · =

∞ X

axn ,

n=0

with a 6= 0. The sum of the first n terms is sn = a + ax + ax2 + · · · + axn−1 . Multiple both sides by x, we have xsn = ax + ax2 + ax3 + · · · + axn−1 + axn . 54

3.1. INTRODUCTION TO METRIC SPACE Subtracting the above two equations gives sn − xsn = a − axn , or

a(1 − xn ) . 1−x The series converges if −1 < x < 1, that is, sn =

∞ X

a a(1 − xn ) = . n→∞ 1−x 1−x

axn = lim sn = lim

n=0

n→∞

(3.3)

An important application in economics is to calculate the present value of an infinite stream of unit incomes. Let a = 1 and x = 1/(1 + r) where r is the interest rate in each period, (3.3) implies that ∞ X n=0

1 1+r = . (1 + r)n r

You should be able to show that ∞ X n=1

1 1 = . n (1 + r) r

P Example 3.5. The harmonic series ∞ n=1 1/n show that the condition that an → 0 is not sufficient for convergence. We have ∞ X 1 1 1 1 = 1 + + + + ··· n 2 3 4 n=1       1 1 1 1 1 1 1 1 1 = 1+ + + + + + + + + ··· + 2 3 4 5 6 7 8 9 16       1 1 1 1 1 1 1 1 1 ≥ 1+ + + + + + + + + ··· + 2 4 4 8 8 8 8 16 16 1 1 1 = 1 + + + + ··· . 2 2 2

Therefore the series diverges. 55

CHAPTER 3. BASIC TOPOLOGY y 4 3 2 1

−4

−3

−2

−1

0

1

x

Figure 3.1: The Exponential Function y = ex .

Example 3.6. Another interesting series is ∞ X 1 e= , n! n=0

where n! = n(n − 1) · · · (2)(1) and 0! = 1. Observe that ∞ X 1 1 1 1 = 1+1+ + + + ··· n! (2)(1) (3)(2)(1) (4)(3)(2)(1) n=0

< 1+1+

1 1 1 + 2 + 3 + ··· 2 2 2

= 3, where we have applied the geometric series in (3.3) to the last equality with a = 1 and x = 1/2. Therefore the series converges to a number that we call e. It can be shown that e is irrational. An alternative definition is  n 1 e = lim 1 + . n→∞ n

56

3.1. INTRODUCTION TO METRIC SPACE Example 3.7. For every real number x, we define the exponential function as2 x

e =

∞ X xn n=0

n!

.

It can be shown that the series converges for every real number so that the function is everywhere defined in R. The function is increasing so it is one-to-one. Some of the important properties of the exponential function is listed below. For all x, y ∈ R, 1. ex > 0, e0 = 1; 2. ex → ∞ as x → ∞, ex → 0 as x → −∞ (see figure 3.1); 3. ex+y = ex ey ; 4. the function is differentiable in R and d(ex )/ dx = ex ; 5. the function increases faster than any power function of x, that is, xn = 0, lim x→∞ ex for all n ∈ N; 6. the function is a bijection from R to R++ , the inverse function is the natural log function, f (y) = log y. Example 3.8. Most mathematicians agree that the series ∞ X

n = 1 + 2 + 3 + ···

n=1

diverges to infinity. By exploiting the paradoxes of infinity, some scientists claim that the series converges to −1/12. Overbye (2014) provides an entertaining description of their claim. 2

The exponential function is also defined for complex numbers. But we shall not pursue the analysis here.

57

CHAPTER 3. BASIC TOPOLOGY

3.2

Continuous Functions

Suppose f is a function from a metric space X to a metric space Y . Let E ⊆ X and p ∈ E. Then f is continuous at p if for every  > 0 there exists a δ > 0 such that ρX (x, p) < δ implies that ρY (f (x), f (p)) <  for all x ∈ E. If f is continuous at every point in E, then f is said to be continuous on E. A continuous bijection of two metric spaces is called a homeomorphism. Example 3.9. We show that the function f : R → R given by f (x) = ex with the Euclidean metric is continuous. That is, we have to show that for any p ∈ R and  > 0, we can find a δ > 0 such that whenever ρX (x, p) = |x − p| < δ, we have ρY (f (x), f (p)) = |ex − ep | < . To this end, notice that ex is an increasing function. If x is between log(ep − ) and log(ep + ), then ex is between ep −  and ep + . Therefore we choose δ = min{p − log(ep − ), log(ep + ) − p}. Then |ex − ep | <  as required.

3.2.1

Properties of Continuous Functions

There are several useful characterizations of a continuous function: Theorem 3.2. Let f be a function which maps a metric space X into a metric space Y . The following statements are equivalent: 1. The function f is continuous on X. 2. For every open set V ⊆ Y , the pre-image f −1 (V ) is open in X. 3. For every closed set W ⊆ Y , the pre-image f −1 (W ) is closed. 58

3.2. CONTINUOUS FUNCTIONS 4. Let {xn } be any sequence in X which converges to a point x. Then the sequence {f (xn )} converges to f (x) in Y . Proof. (1 ⇔ 2) Suppose f is continuous on X and V ⊆ Y is open. Our goal is to show that every point in f −1 (V ) is an interior point. So suppose p ∈ f −1 (V ) so that f (p) ∈ V . Since V is open, there exists an open ball B (f (p)) ⊆ V . In other words, for every point y ∈ Y such that ρY (f (p), y) < , y ∈ V . On the other hand, since f is continuous at p, there exist a δ > 0 such that ρY (f (p), f (x)) <  whenever ρX (p, x) < δ. This means that Bδ (p) ⊆ f −1 (V ). Hence p is an interior point of f −1 (V ). Conversely, suppose f −1 (V ) ⊆ X is open for every open set V ⊆ Y . Let p ∈ X and  > 0. Let V = B (f (p)). Since an open ball is open, f −1 (V ) is also open. This implies that p is an interior point of f −1 (V ). By definition there exists δ > 0 such that ρX (p, x) < δ implies that x ∈ f −1 (V ). It follows that f (x) ∈ V , or ρY (f (x), f (p)) < . This shows that f is continuous at p. (2 ⇔ 3) W is closed if and only if W c is open so that f −1 (W c ) is open by (2). The proof of statement (4) is left as an exercise. Continuity preserves compactness and connectedness: Theorem 3.3. Let f be a continuous function from a metric space X to a metric space Y . 1. If E ⊆ X is compact, then f (E) is also compact. 2. If E ⊆ X is connected, then f (E) is connected. Proof. Proof of part 1 can be found in Rudin (1976, p. 89). Here we prove part 2 by contradiction. Assume that E is connected but f (E) is separated, that is, f (E) = A ∪ B, where A, B ⊂ Y are non-empty and separated. Let G = E ∩ f −1 (A) and H = E ∩ f −1 (B). Notice that E = G ∪ H and both G and H are non-empty sets. ¯ By the above corollary, A¯ is closed implies that Since G ⊆ f −1 (A), G ⊆ f −1 (A). −1 ¯ ¯ ⊆ f −1 (A), ¯ or f (G) ¯ ⊆ A. ¯ Since f (H) = B and f (A) is also closed. Therefore G ¯ ∩ f (H) = ∅. This implies that G ¯ ∩ H = ∅, otherwise any A¯ ∩ B = ∅, we have f (G) ¯ ∩ H will have f (x) ∈ f (G) ¯ ∩ f (H). x∈G ¯ = ∅. Together these mean By a similar argument we can show that G ∩ H that G and H are separated, which is a contradiction since E = G ∪ H and E is connected. 59

CHAPTER 3. BASIC TOPOLOGY If f is a functional, then the compactness property guarantees that f has a maximum value and a minimum value. The following result is sometimes called Weierstrass Theorem. Theorem 3.4. Let f be a continuous functional on a metric space X. If E ⊆ X is compact, then there exist p and q in E such that for all x ∈ E, f (q) ≤ f (x) ≤ f (p). Proof. Let M = sup f (E) and m = inf f (E). Since E is compact, then by theorem 3.3 f (E) is compact. It follows that M and m are in f (E) and the result follows. The connectedness property leads to the important intermediate-value property of continuous functionals. Theorem 3.5 (Intermediate Value Theorem). Let f be a continous functional on the interval [a, b]. Suppose that f (a) < f (b). For every number c in the interval (f (a), f (b)), there exists a number x ∈ (a, b) such that f (x) = c. You should be able to proof the following results from the definition of continuity. Theorem 3.6. Let f and g be continuous functionals on a metric space X and α ∈ R. Then αf, f + g, f g, g ◦ f , and f /g, provided that g(x) 6= 0 for all x ∈ X, are continuous on X.

3.2.2

Semicontinuity

Theorem 3.4 implies that solutions for optimization problems on a compact set always exist if the objective function is continuous. In a typical economic application, however, we only need to find either the maximum or the minimum but not both. In these situations we can relax the continuity requirement a little bit. For example, for a maximum solution to exist, we require the objective function to be upper semicontinuous. Formally, a functional f on a metric space X is upper semicontinuous if for every α ∈ R, the upper contour set {x ∈ X : f (x) ≥ α} 60

3.2. CONTINUOUS FUNCTIONS y

y







• x

x0

x0

(a) An Upper Semicontinuous Function

x

(b) A Lower Semicontinuous Function

Figure 3.2: Semicontinuous Functions is closed. Similarly, f is lower semicontinuous if the lower contour set {x ∈ X : f (x) ≤ α} is closed for all α ∈ R. In Figure 3.2(a), for any value of α between the jump at the point x0 , the upper contour set defined above is closed but the lower contour set is not. Therefore the function is upper semicontinuous but not lower semicontinuous. You should convince yourself that the function in Figure 3.2(b) is lower semicontinuous. It is straight forward to show that a function is continuous if and only if it is both upper and lower semicontinuous.

3.2.3

Uniform Continuity

Notice that in the definition of continuity above, δ depends on  and the point p. The following definition put a stricter condition on f : A function f : X → Y is uniformly continuous on E ⊆ X if for all  > 0 there exists a δ > 0 such that if p, q ∈ E and ρX (p, q) < δ, then ρY (f (p), f (q)) < . Thus δ depends only on  and not on p or q. It is obvious that a uniformly continuous function is continuous. We state the following useful result without proof (see Rosenlicht, 1976, 80–81): Theorem 3.7. Let f be a continuous function from a metric space X to a metric 61

CHAPTER 3. BASIC TOPOLOGY

Luitzen Egbertus Jan Brouwer (1881–1966)

space Y . If E ⊆ X is compact, then f is uniformly continuous on E. Theorem 3.2 guarantees that the image of a convergent sequence is also convergent under a continuous mapping. The following example shows that this is not necessary so for a Cauchy sequence. Example 3.10. Define the continuous function f (x) = log x on R++ = (0, ∞). Consider the Cauchy sequence {xn } = {e−n }. The image of the sequence with f is {yn } = {−n}, which is not a Cauchy sequence. The following two results provide the remedies. Theorem 3.8. Let f be a uniformly continuous function which maps a metric space X into a metric space Y and let {xn } be a Cauchy sequence in X. Then the image {f (xn )} is also a Cauchy sequence in Y . Theorem 3.9. Let X be a complete metric space. Then E ⊆ X is also a complete metric space if and only if it is closed. In example 3.10, f is not uniformly continuous on R++ . On the other hand, R++ is not complete and so the Cauchy sequence {xn } = {e−n } is not convergent.

3.2.4

Fixed Point Theorems

Let X be a metric space and f : X → X. Then x ∈ X is called a fixed point if f (x) = x. In economics a fixed point is an equilibrium point in a dynamical system. Therefore the conditions of existence of fixed point is important. We illustrated the problem with a simple example. 62

3.3. SEQUENCES AND SERIES OF FUNCTIONS Suppose that f : [a, b] → [a, b] is continuous. Then there exists at least one fixed point. To show this, let g(x) = f (x) − x on [a, b]. Then g(a) ≥ 0 ≥ g(b). By theorem 3.5 there exist a point x ∈ [a, b] such that g(x) = 0, which implies that f (x) = x. An extension of this result to a more general setting is as follows. Interested readers can consult McLennan (2014) for extensive discussions on fixed point theorems. Theorem 3.10 (Brouwer Fixed Point Theorem). Let E be an nonempty, compact, and convex subset of an Euclidean space. Then every continuous function f : E → E has at least one fixed point.

3.3

Sequences and Series of Functions

In this section we briefly discuss sequences and series of functions, which sometimes appear in probability theory, approximation of functions, and dynamical systems.

3.3.1

Pointwise Convergence

Let {fn } be a sequence of functionals defined on a set S. Suppose that for every point x ∈ S, the sequence of numbers {fn (x)} converges. Then we can define a limit function f (x) on S such that f (x) = lim fn (x). n→∞

We say that the sequence of function {fn } converges pointwise on S. Similarly, P if the series fn (x) converges for every x ∈ S, then we call the function f (x) =

∞ X

fn (x)

n=1

P the sum of the series fn (x). Naturally, we are interested in whether properties of each of the functions fn will be inherited by the limit function or the sum. The following simple example shows that it is not always the case. Example 3.11. Suppose that fn : [0, 1] → R is defined by fn (x) = xn . 63

CHAPTER 3. BASIC TOPOLOGY Then for 0 ≤ x < 1, fn (x) → 0. When x = 1, fn (x) → 1. Therefore the limit function is  0 if 0 ≤ x < 1, f (x) = 1 if x = 1. Example 3.11 shows that although each of the functions fn in the sequence is continuous on the interval [0, 1], the limit function f (x) is not. In other words, when x = 1, lim lim fn (t) 6= lim lim fn (t). t→x n→∞

n→∞ t→x

It turns out that properties in differentiation and integration are not preserved under pointwise convergence as well. We need a stricter criterion for convergence that can carry over these properties to the limit function.

3.3.2

Uniform Convergence

A sequence of functionals {fn } defined on a set S is said to uniformly converge to a limit function f if for every  > 0, there exists an integer N such that for all n > N, |fn (x) − f (x)| <  (3.4) for all x ∈ S. The key difference between pointwise convergence and uniform convergence is that, in the former N depends on  and x, whereas uniform convergence means that N depends only on . In other words, inequality (3.4) is satisfied uniformly for all x ∈ S once N is chosen. A formal definition for uniform convergence P of the series fn (x) is left as an exercise. Not surprisingly, the Cauchy criterion applies to uniform convergence. Theorem 3.11. A sequence of functionals {fn } defined on a set S uniformly converges to a limit function f on S if and only if for every  > 0, there exists an integer N such that for all m, n > N , |fn (x) − fm (x)| <  for all x ∈ S. Example 3.12. Show that the sequence of functions defined by fn (x) =

x 1 + nx2

64

3.3. SEQUENCES AND SERIES OF FUNCTIONS converges uniformly to a function f for every x ∈ R. Proof. Observe that for every x ∈ R, x |x| 1 + x2 = 1 + x2 < 1. For any  > 0, let N > 1/. Then for all n > N , x |fn (x)| = 1 + nx2   1 |x| = n 1/n + x2   1 |x| ≤ n 1 + x2 1 1 < < < . n N This shows that {fn (x)} converges uniformly to f (x) = 0 on R. ¯ n = (X1 + X2 + · · · + Xn )/n denote Example 3.13 (Central Limit Theorem). Let X the mean of a random sample of size n from a distribution that has mean µ and variance σ 2 . Let √ ¯ n(Xn − µ) . Zn = σ Then {Zn } is a sequence of random variables with probability density functions {fn }. The Central Limit Theorem states that {fn } converges uniformly to the probability density function of a normal distribution with mean 0 and variance 1. That is, the limit function is 1 2 f (x) = √ e−x /2 . 2π This is a remarkable result because any distribution with a finite mean and variance converges to the standard normal distribution, and knowledge of the functional form of fn is not necessary. Theorem 3.12. Suppose that {fn } is a sequence of continuous functions that converges uniformly to a limit function f on a set S. Then f is continuous on S. Interested readers can consult Rudin (1976, chapter 7) for a proof of this result and the relation between uniform convergence and differentiation and integration. For series of functions, there is a test for uniform convergence. 65

CHAPTER 3. BASIC TOPOLOGY Theorem 3.13 (Weierstrass M -Test). Let {fn } be a sequence of functions defined on a set S and let {Mn } be a sequence of real numbers. Suppose that for all x ∈ S and all n ∈ N, |fn (x)| ≤ Mn . P P If Mn converges, then fn (x) converges uniformly on S.

Proof. The proof uses the Cauchy criterion for series of functions. That is, suppose P Mn converges. Then for any  > 0, there exists an integer N such that for all m, n > N , n n X X Mi < , fi (x) ≤ i=m i=m P for all x ∈ S. Therefore fn (x) converges uniformly on S.

Example 3.14. In a basic centralized model in macroeconomics, the economy seeks P t to maximized a series of utility function ∞ t=1 β U (ct ), where U is the instantaneous utility function of consumption ct , and β ∈ (0, 1) is a discount factor. We use t as the counter instead of n to reflect that it represents time periods. The constraint of the economy in each period is F (kt ) = ct + kt+1 − (1 − δ)kt ,

(3.5)

where F is the aggregate production function, kt is the capital stock, and δ is the depreciation rate of capital. The function F is usually assumed to be increasing and concave, reflecting decreasing returns to scale. Then the economy will converge to an equilibrium kt = k ∗ . From (3.5) the maximum consumption is cM = F (k ∗ ), which means that the economy consume the whole output and makes zero investment, that is, kt+1 − (1 − δ)kt = 0. P Define Mt = β t U (cM ). Then the geometric series Mt converges. Since by deft inition β U (ct ) ≤ Mt , for all t, by theorem 3.13 the series of utility functions P∞ t t=1 β U (ct ) converges uniformly to a limit function. If the instantaneous utility function U is continuous, then by theorem 3.12 the limit function is continuous. It follows from theorem 3.4 that a maximum value exists. The technique of dynamic optimization will be discussed in chapter 9. 66

3.4. EXERCISES

3.4

Exercises

1. Let S = {a, b, c}. Show that T = {∅, {a, b}, {b}, {b, c}, S} is a topology on S. 2. Let X be an infinite set. For x, y ∈ X, define  1 if x 6= y, ρ(x, y) = 0 if x = y. Prove that (X, ρ) is a metric space. Give an example of proper subset of X that is open. Are there any closed sets and compact sets? 3. Show that the set Rn with the distance function ρ(x, y) = maxni=1 |xi − yi | is a metric space. 4. (Symbolic dynamics) Suppose that the set Ω2 contains all bi-infinite sequences of two symbols, say 1 and 2. A point a ∈ Ω2 is the sequence a = {. . . , a−2 , a−1 , a0 , a1 , a2 , . . . }, where ai = {1, 2}. The position of a0 must be specified. Otherwise, a sequence such as a = {. . . , 2, 1, 2, 1, 2, 1, . . . } can represent two distinct sequences, one with a0 = 2 and the other with a0 = 1. The metric between two points a, b ∈ Ω2 is defined by ρ(a, b) = where

∞ X δ(ai , bi ) , |i| 2 i=−∞

 0 if a = b , i i δ(ai , bi ) = 1 if ai = 6 bi .

Show that (Ω2 , ρ) is a metric space. 67

CHAPTER 3. BASIC TOPOLOGY 5. Determine whether the following sets are (i) open, (ii) closed, (iii) both open and closed, or (iv) neither open nor closed: (a) Z ⊆ R (b) {(x, y) : 1 < x < 2, y = x} ⊆ R2 , (c) {x ∈ Q : 1 ≤ x ≤ 2} ⊆ R 6. Prove or disprove: Let A and B be connected in a metric space. Then there exists a boundary point in one of the sets which belongs to the other set. 7. Consider the Euclidean metric on R. Let A = (−1, 1) ∪ (1, 2), that is, A is the union of two intervals. Answer the following questions with explanations. (a) What is the diameter of A? (b) Is A open, closed, or neither? (c) Is the point x = 1 a limit point of A? ¯ (d) What is A? (e) Is A connected? 8. Consider the Euclidean metric space R2 . Let A = {x ∈ R2 : ρ(x, 0) ≤ 1}. (a) What is the diameter of A? (b) What is b(A)? ¯ (c) What is A? (d) Is A bounded? (e) Is A open, closed, neither, or both? (f) Is A perfect, separated, compact, and convex? 9. Let R be the set of real numbers. Define a metric ρ on R as ( 1 if x 6= y, ρ(x, y) = 0 if x = y. Let A = {1/n : n = 1, 2, 3, . . .} and B = {−4, e, π}. Answer the following questions with explanation. (a) Is the point 0 a limit point of A? 68

3.4. EXERCISES (b) Is A an open set, closed set, both, or neither? (c) Is B an open set, closed set, both, or neither? (d) What is the diameter of B? 10. Consider the Euclidean space R2 . Let A = {(x, 0) : a < x < b}, that is, A is an open interval on the horizontal axis. Determine and explain whether A is (a) open, (b) closed, (c) perfect, (d) compact. 11. Prove that a set is closed if and only if its complement is open. 12. Show that (a) the intersection of two closed sets is closed, (b) the union of two closed sets is closed, (c) the intersection of two open sets is open, (d) the union of two open sets is open. 13. Let (X, ρ) be a metric space. Suppose that A and B are separated subsets of X. Prove or disprove: A¯ ⊆ B c . 14. Define the sequence {xn } as {1, 1/2, 1/4, 1/6, . . .}. (a) What is the range of {xn }? (b) Is {xn } bounded? (c) Let A be the range of {xn }. Find sup A. (d) Find inf A. 15. Prove or disprove that the following sequences {xn } in R converge: √ (a) xn = 1/ n. (b) xn = n2 . 69

CHAPTER 3. BASIC TOPOLOGY (c) xn = (−1)n . (d) xn = 1 + (−1)n /n. In each case determine if the sequence is bounded and if the range is finite. 16. Let {xn } be sequence in a metric space (X, ρ) which converges to a point a. Let A denotes the range of {xn }. Is a always a limit point of the set A? 17. Show that the limit of a sequence is unique. 18. Let {xn } and {yn } be two sequences in a metric space (X, ρ). (a) Show that limn→∞ xn = x if and only if for all r > 0, the open ball Br (x) contains all but finitely many terms of {xn }. (b) Suppose limn→∞ xn = x and ρ(xn , yn ) → 0. Show that limn→∞ yn = x. (c) Consider the case that X = R so that the metric is ρ(x, y) = |x−y|. Suppose limn→∞ xn = x and limn→∞ yn = y. Show that limn→∞ (xn + yn ) = x + y. 19. Show that a convergent sequence in a metric space is a Cauchy sequence. 20. Consider the natural order ≥ on the set of rational numbers Q. Define a sequence {xn } in Q as follows: x1 = 3, x2 = 3.1, x3 = 3.14, x4 = 3.141, x5 = 3.1415, x6 = 3.14159, .. . That is, xn+1 includes one more digit from the number π than xn . Note that π∈ / Q. Let A be the range of {xn }. (a) What is the set of upper bounds of A? (b) Does sup A exists in Q? If yes, what is it? (c) Does inf A exists in Q? If yes, what is it? 70

3.4. EXERCISES 21. Let {xn } be an increasing sequence. Suppose {xn } is bounded. Prove that the sequence converges by following the hints below: (a) Let E be the range of {xn } and let x = sup E. (b) Show that for every  > 0, there exists an integer N such that x− < xN < x. (c) Now since {xn } is increasing, n > N implies that x −  < xn < x. 22. Find

n ∞  X r , 1 + r n=0

where r is the market interest rate. 23. Show that the function f : R → R given by f (x) = x2 with the Euclidean metric is continuous. Hint: you may find this useful: |x2 − p2 | = |(x + p)(x − p)| = |(x − p + 2p)(x − p)| ≤ (|x − p| + 2|p|)|x − p|. 24. Show that the rotation function defined in example 2.28 of chapter 2 is continuous. 25. Show that the function f : R → R given by  1 if x is rational, f (x) = 0 if x is irrational, is nowhere continuous. ¯r (p) = {x ∈ X : ρ(x, p) ≤ r} be a closed ball centred at p with radius r in 26. Let B ¯r (p)) is a metric space X. Let f be a continuous functional on X. Show that f (B bounded. 27. Suppose that f : X → R is continuous. Prove that %f (a) = {x : f (x) ≥ a} is closed. 28. Suppose that S is a compact set in a metric space X. Show that every continuous functional f : S → R is bounded, that is, f (S) is bounded. 71

CHAPTER 3. BASIC TOPOLOGY 29. Let u : X → R be a utility function on an ordered set (X, %). Let g : R → R be a strictly increasing function. Show that g ◦ u is also a utility function representing the order %. 30. Let f : R → R be defined by  x2 if x 6= 1, f (x) = 10 if x = 1. (a) Is f lower semi-continuous? (b) Is f upper semi-continuous? Provide brief explanations to your answers. 31. Suppose that f : R2 → R is given by  x 2 + y 2 f (x, y) = 2

if (x, y) 6= (0, 0), if (x, y) = (0, 0).

(a) Define the graph of f . (b) Is f upper semi-continuous? Explain. (c) Is f lower semi-continuous? 32. Suppose X, Y, Z are metric spaces and f : X → Y and g : Y → Z are continuous functions. Prove that g ◦ f : X → Z is continuous. 33. Prove theorem 3.8. 34. Prove theorem 3.9. 35. Let S = [0, 1] ∈ R. Show that a continuous function f : S → S have at least one fixed point. P 36. Give a formal definition of uniform convergence for the series of functions fn (x) on a set S. 37. Show that the sequence of functions {fn } where fn (x) = 72

1 1 + xn

3.4. EXERCISES converges to a limit function on S = [0, 1]. Does it converge uniformly? 38. Show that

cos nx fn (x) = √ n

converges uniformly to a limit function.

73

CHAPTER 3. BASIC TOPOLOGY

74

Chapter 4 Linear Algebra Linear algebra is the starting point of multivariate analysis, due to its analytical and computational simplicity. In many economic applications, a linear model is often adequate. Even in a more realistic nonlinear model, variables are linearized at the point of interest, usually at a steady-state equilibrium, to study their behaviours in the neighbourhood. In this chapter we first study the basic properties of vector spaces. Then we turn our attention to transformations between vector spaces.

4.1

Vector Spaces

In chapter 3 we study “distance” between objects in a set, the behaviour of sequences of those objects, and functions between spaces that preserve proximity. A vector space is an abstract idea of combining objects in a set in a linear fashion. The objects are called vectors, and by linear combination we mean that we can scale vectors up or down by multiplying them with scalers, usually real or complex numbers, and then combine them to form another vector. In this chapter we take the scalers to be real numbers. Formally, a vector space V is a set of objects called vectors with two operations. The first is called vector addition, which combines two vectors to form a third one. That is, for any x, y ∈ V , we call x + y ∈ V the sum of x and y. The second operation is scalar multiplication, which multiplies a real number to a vector to form another vector. That is, for any x ∈ V and α ∈ R, the product αx is in V . Vector additions and scalar multiplications in a vector space follow eight rules or axioms. For any x, y, z ∈ V and α, β ∈ R: 75

CHAPTER 4. LINEAR ALGEBRA 1. Commutative in addition: x + y = y + x. 2. Associative in addition: (x + y) + z = x + (y + z). 3. There exists a unique zero vector 0 ∈ V such that x + 0 = x. 4. For any x ∈ V , there exist a vector −x ∈ V such that x + (−x) = 0. 5. 1x = x. 6. (αβ)x = α(βx). 7. α(x + y) = αx + αy. 8. (α + β)x = αx + βx. Axioms 1 and 2 imply that the order of vector addition is not important. For example, x + y + z = z + x + y. In axiom 8, the addition symbol on the left hand side is for real numbers, while the one on the right is for vectors. Example 4.1. Let V = Rn . For any n-tuples x, y ∈ Rn , where x = (x1 , x2 , . . . , xn ), y = (y1 , y2 , . . . , yn ), vector addition is defined by x + y = (x1 + y1 , x2 + y2 , . . . , xn + yn ). For any α ∈ R, scalar multiplication is defined by αx = (αx1 , αx2 , . . . , αxn ). It is straight forward to verify that the definitions satisfy the eight axioms. Figure 4.1 illustrates what we learned in high school about vector addition. Vectors are represented as arrows, which are defined by their length and direction. To find a+b, we move the tail of vector b to the head of vector a, while keeping the direction unchanged. The vector from the tail of a to the head of b is a + b. If we instead put 76

4.1. VECTOR SPACES

a

b

a+b=b+a b

a

Figure 4.1: Vector Addition in R2 the tail of a on the head of b, we get b + a. And as the diagram shows, the results confirm axiom 1, that vector additions are commutative. The scalar multiplication αa, on the other hand, means that the length of a is multiplied by α. If α is negative, then αa and a have opposite directions. Example 4.2. Let V = P (n) be the set of all polynomials of degree n. For any f, g ∈ P (n), we have f (x) = a0 + a1 x + a2 x2 + · · · + an xn , g(x) = b0 + b1 x + b2 x2 + · · · + bn xn , where ai , bi ∈ R, i = 0, 1 . . . , n. The sum f + g is defined by (f + g)(x) = (a0 + b0 ) + (a1 + b1 )x + (a2 + b2 )x2 + · · · + (an + bn )xn . Scalar multiplication is defined by αf ∈ P (n) such that (αf )(x) = (αa0 ) + (αa1 )x + (αa2 )x2 + · · · + (αan )xn . In fact, the operations are the same as those in example 4.1 if we let (a0 , a1 , . . . , an ) and (b0 , b1 , . . . , bn ) be (n + 1)-tuples. Example 4.3. Let V = S be the set of all sequences in R. For all {xn }, {yn } ∈ S and α ∈ R, we can define {xn } + {yn } = {xn + yn }, and α{xn } = {αxn }. 77

CHAPTER 4. LINEAR ALGEBRA Then S is a vector space. If we narrow the definition of S to the set of convergent sequences, is it a vector space?

Example 4.4. Define V = Mm×n as the set of all matrices on real numbers with m rows and n columns. That is, any A ∈ Mm×n is in the form   a11 a12 · · · a1n    a21 a22 · · · a2n  A= .. ..  ..  .. . . . . .   am1 am2 · · ·

amn

Vector addition is simply matrix addition:    b11 b12 · · · a11 a12 · · · a1n     a21 a22 · · · a2n   b21 b22 · · ·  A+B = .. .. ..  ... ... + .  .. . .   .. .  . bm1 bm2 · · · am1 am2 · · · amn   a11 + b11 a12 + b12 · · · a1n + b1n    a21 + b21 a22 + b22 · · · a2n + b2n  . = .. .. .. ..   . . . .   am1 + bm1 am2 + bm2 · · ·

 b1n  b2n  ..   .  bmn

amn + bmn

And scalar multiplication is 

αa11   αa21 αA =   ..  .

αa12 αa22 .. .

··· ··· .. .

αam1 αam2 · · ·

 αa1n  αa2n  ..  . . 

αamn

In fact, the operations are analogous to those of mn-tuples.

Example 4.5. Let V = F (S, R) be the set of all functionals on a set S. For all f, g ∈ F (S, R), vector addition is defined by (f + g)(x) = f (x) + g(x), 78

4.1. VECTOR SPACES for all x ∈ S. Scalar multiplication is defined by (αf )(x) = (α)f (x). From the above eight axioms, we can establish a few basic results in vector spaces. Let us label the axioms as A1 to A8. Now, if α is a scalar and 0 is the zero vector, then by A3 and A7 α0 = α(0 + 0) = α0 + α0. Adding −(α0) to both sides and using A4, we get α0 = 0.

(4.1)

That is, any scalar multiplication of the zero vector gives you the zero vector. Similarly, we can show that 0x = 0. (4.2) That is, any vector multiplied by the scalar 0 becomes the zero vector. Observe that 0x = (0 + 0)x = 0x + 0x by (A8). Adding the vector −0x to both sides gives you −0x + 0x = −0x + 0x + 0x, which reduces to 0 = 0 + 0x by (A4). Applying (A3) to the right-hand of the equation gives (4.2). Next we show that if αx = 0 then either α = 0 or x = 0. If α 6= 0, then by (4.1) 1 (αx) = 0. α But by A5 and A6 1 (αx) = α



 1 α x = 1x = x. α

Therefore we conclude that x = 0. Another result is that for every vector x ∈ V , (−α)x = −(αx). 79

(4.3)

CHAPTER 4. LINEAR ALGEBRA The reason is 0 = 0x (by (4.2)) = (α − α)x = αx + (−α)x (using A8) and by A4 the result in (4.3) follows. Consequently, we can write y + (−x) as y − x without ambiguity.

Theorem 4.1. Suppose x ∈ V and x 6= 0. If αx = βx, then α = β.

Proof. By A4 there exist a vector called −αx. So adding this vector to both sides of αx = βx gives αx + (−αx) = βx + (−αx). (4.4) It follows that 0 = αx + (−αx) (by A4) = βx + (−αx) (from (4.4)) = βx + (−α)x (using (4.3)) = (β − α)x

(by A8)

Since x 6= 0, by the result above β − α must be 0, from which we conclude that α = β.

4.2

Basic Properties of Vector Spaces

In this section we characterize some common properties of vector spaces. Suppose that A = {x1 , x2 , . . . , xn } is a set of vectors in a vector space V . Then y ∈ V is a linear combination of A if there exists a set of scalars {α1 , α2 , . . . , αn } such that y = α1 x1 + α2 x2 + · · · + αn xn =

n X i=1

80

α i xi .

4.2. BASIC PROPERTIES OF VECTOR SPACES The set of all linear combination of A is called the linear span or linear hull of A. That is, we define ( n ) X span(A) = αi xi ∈ V : αi ∈ R, i = 1, 2, . . . , n . i=1

In general, the set A can be countable or uncountable. Example 4.6. Suppose that V = R3 . Let A = {(1, 0, 0), (0, 3, 0)}. Then span(A) is the x-y plane in the three-dimensional space. That is, span(A) = {(x, y, 0) ∈ R3 : x, y ∈ R}. This is because for any (x, y, 0) ∈ R3 , there exists α, β ∈ R such that (x, y, 0) = α(1, 0, 0) + β(0, 3, 0). Example 4.7. Consider the vector space P (n) of all n-degree polynomials in example 4.2. Let B = {f0 , f1 , . . . , fn } be the subset of polynomials defined by fi (x) = xi ,

i = 0, 1, . . . , n.

Then span(B) = P (n). In this case we say that the set B spans P (n). Let S be a nonempty subset of a vector space V . We call S a subspace of V if S satisfies the definition of a vector space. Recall that any nonempty subset of a metric space is itself a metric space. This, however, does not work in a vector space. This is because any subspace of a vector space has to be “closed” under vector addition and scalar multiplication, as the following theorem shows. Theorem 4.2. A set S is a subspace of a vector space V if and only if for any vectors x, y ∈ S and and any scalar α, the linear combination αx + y is in S. Proof. Suppose that S is a subspace of V . Since by definition any vector x and y in S satisfy the axioms of a vector space, the vector αx + y is in S. Conversely, suppose that S is a nonempty subset of V and, for any vectors x, y ∈ S and and any scalar α, αx + y is in S. Axioms 1 and 2 are by definition satisfied. For any x ∈ S, let α = −1 so that (−1)x + x = −x + x = 0 is in S. Therefore axioms 3 and 4 are satisfied. Axioms 5 to 8 are inherited from V . Therefore S is itself a vector space. 81

CHAPTER 4. LINEAR ALGEBRA Example 4.8. The following are examples of subspaces. 1. Let x be a nonzero vector in a vector space V . Then the set S = {αx : α ∈ R} is a subspace of V . In fact, the linear span of any nonempty subset of V is a subspace of V . 2. The set P (n) of all n-degree polynomials defined in example 4.2 is a subspace of the vector space F (S, R) defined in example 4.5 with S = R. 3. Let S be the vector space of all sequences in R defined in example 4.3. Then the set of all convergent sequences in R is a subspace of S. Let A and B be any nonempty subsets of a vector space V . Then the Minkowski sum, or simply the sum of sets A and B is defined as A + B = {a + b : a ∈ A, b ∈ B}. The definition can be extended to any number of subsets. Example 4.9. In example 2.9, chapter 2 we define the production set of a firm. Suppose that an economy has n commodities and l firms. The firms’ production sets are represented by Yj ∈ Rn , j = 1, . . . , l. In the analysis of the whole economy, we can add up the production of all firms into an aggregate firm, with the aggregate production set l X Y = Y1 + Y2 + · · · + Yl = Yj . j=1

A subset A = {x1 , x2 , . . . , xn } of a vector space V is said to be linearly dependent if there exists a set of scalars {α1 , α2 , . . . , αn }, not all zeros, such that α1 x1 + α2 x2 + · · · + αn xn = 0.

(4.5)

That is, the zero vector is a linear combination of A with some nonzero scalars. Otherwise, the set A is called linearly independent. Alternatively, we can define a set of vectors A = {x1 , x2 , . . . , xn } to be linearly independent if the condition in equation (4.5) implies that αi = 0 for i = 1, 2, . . . , n. You can verify the following consequences readily from the definitions: 82

4.2. BASIC PROPERTIES OF VECTOR SPACES 1. If a set A contains the zero vector, then it is linearly dependent. 2. Any set containing a linearly dependent set is linearly dependent. 3. Any subset of a linearly independent set is linearly independent. Let B be a linearly independent set of vectors of a vector space V . Then B is called a basis for V if span(B) = V . If B is finite, we call the cardinality of B the dimension of V . For example, if |B| = n, we write dim V = n. If B is a basis for V , any vector x ∈ V can be expressed as a linear combination of the vectors in B, that is, x = α1 x1 + α2 x2 + · · · + αn xn , for some scalars αi , i = 1, 2, . . . , n. We define the n-tuple (α1 , α2 , . . . , αn ) formed by the scalars of the linear combination the coordinates of x relative to the basis B. Example 4.10. Consider the vectors (1, 2) and (2, 1) in R2 . If α(1, 2) + β(2, 1) = (0, 0), then α + 2β = 0, 2α + β = 0, which gives α = β = 0. Therefore the two vectors are linearly independent. For any vector (x1 , x2 ) ∈ R2 , we can write (x1 , x2 ) = α(1, 2) + β(2, 1), which implies that α + 2β = x1 , 2α + β = x2 . Solving the system of equation gives α=

−x1 + 2x2 , 3 83

β=

2x1 − x2 . 3

CHAPTER 4. LINEAR ALGEBRA The two vectors span R2 and therefore form a basis for the vector space. Example 4.11. Consider the following vectors in Rn : e1 = (1, 0, 0, . . . , 0), e2 = (0, 1, 0, . . . , 0), .. . en = (0, 0, 0, . . . , 1). The set B = {e1 , e2 , . . . , en } is a basis for Rn . Let x = (x1 , x2 , . . . , xn ) be any vector in Rn . Then x = x 1 e 1 + x2 e 2 + · · · + xn e n . Therefore (x1 , x2 , . . . , xn ) is the coordinates of x relative to the basis B. In this case, the components of x coincide with the coordinates. For this reason we call {e1 , e2 , . . . , en } the standard basis of Rn . It is clear from examples 4.10 and 4.11 that the coordinates of a vector change with the choice of the basis for a vector space. We summarize some important facts on basis and dimension of a vector space as follows. 1. If B is a finite set of linearly independent vectors that spans the vector space V , then any other basis for V must have the same cardinality as B. That is, dim V is a unique number. 2. If dim V = n, then any set containing more than n vectors in V is linearly dependent. Any set containing less than n vectors cannot span V . 3. If dim V = n and W is a proper subspace of V , then dim W < n. 4. The set {0} is called the zero subspace of V . But by definition the zero subspace is not linearly independent. Therefore we define dim{0} = 0. Example 4.12. You can verify the dimensions of the vector spaces in examples 4.1 to 4.5 by identifying their standard basis: • dim Rn = n, • dim P (n) = n + 1, • S is infinite-dimensional, 84

4.2. BASIC PROPERTIES OF VECTOR SPACES • dim Mm×n = m × n,

• F (S, R) is infinite-dimensional.

In example 4.7 we show that the set B = {f0 , f1 , . . . , fn } where fi (x) = xi ,

i = 0, 1, . . . , n,

is a basis for the vector space P (n) of all n-degree polynomials. In this case the order of the set B is somewhat important because polynomials are expressed in increasing power of x. For this reason we define an ordered basis as an n-tuple of independent vectors in the desirable order, B = (f0 , f1 , . . . , fn ). Similarly, the standard ordered basis for Rn is B = (e1 , e2 , . . . , en ).

Example 4.13. In matrix analysis, vectors in Rn are written as Mn×1 column matrices to conform with matrix multiplication. Let A be an n × n invertible matrix. The column vectors of A, denoted by a1 , a2 , . . . , an ∈ Rn , form a basis for Rn . This is because for any vector x in Rn , we can define a vector α = (α1 , α2 , . . . , αn ) such that, written in matrix form, x = Aα, or x = α1 a1 + α2 a2 + · · · + αn an . Therefore x can be expressed as a linear combination of {a1 , a2 , . . . , an }. The coordinate α of x can be solved by the inverse of A, that is, α = A−1 x. In example 4.10, A=

! 1 2 . 2 1

The readers can verify the values of α and β using equation (4.6). 85

(4.6)

CHAPTER 4. LINEAR ALGEBRA

4.3 4.3.1

Linear Transformations Introduction

A linear transformation is a function f which maps a vector space V into a vector space W such that for all x, y ∈ V and scalar α ∈ R, f (αx + y) = αf (x) + f (y). That is, f is additive and linearly homogeneous. If f maps a vector space V into itself, it is call a linear operator. The pre-image of the zero vector in W , f −1 (0) is called the kernel or null space of f . It can be shown that the kernel of f is a subspace of V . The dimension of the kernel of f is called the nullity of f . Also, the range of f , f (V ) is a subspace of W . If V is finite-dimensional, the dimension of f (V ) is called the rank of f . The following theorem relates the rank and nullity of f . Theorem 4.3 (Dimension Theorem). Let f : V → W be a linear transformation. Suppose that V is finite-dimensional. Then rank f + nullity f = dim V. A proof of this important result can be found in Hoffman and Kunze (1971, 71–72). A transformation f is said to have full rank if rank f = min{dim V, dim W }. The set of all linear transformations from V into W can be denoted by L(V, W ). For every f, g ∈ L(V, W ) and α ∈ R, define 1. (f + g)(x) = f (x) + g(x), 2. (αf )(x) = α(f (x)). It is straight forward to show that f + g and αf are linear transformations. In fact L(V, W ) is a vector space, with dim L(V, W ) = dim V × dim W . 86

4.3. LINEAR TRANSFORMATIONS

4.3.2

Matrix Representations of Transformations

Let V be an n-dimensional vector space and W a m-dimensional vector space. Suppose that BV = (v1 , . . . , vn ) is an ordered basis of V and BW = (w1 , . . . , wm ) is an ordered basis of W . Let f be a linear transformation from V into W . Then f can be defined by its action on the vectors vj . In particular, f (vj ) can be expressed as a linear combination of the vectors in BW , that is, f (vj ) = a1j w1 + a2j w2 + · · · + amj wm , where (a1j , . . . , amj ) are the coordinates of f (vj ) relative to the basis BW . For j = 1, . . . , n, the m × n scalars aij can be arranged in the form of a matrix 

a11   a21 A=  ..  .

a12 a22 .. .

··· ···

am1 am2 · · ·

 a1n  a2n  ..  . . 

amn

That is, we write the coordinates of f (vj ) as the j-th column in A. A is called the matrix representation of f relative to the ordered bases BV and BW . Since BV is a basis of V , any vector x ∈ V can be expressed as a linear combination x = x 1 v 1 + · · · + xn v n , 87

CHAPTER 4. LINEAR ALGEBRA where (x1 , . . . , xn ) are the coordinates of x relative to BV . Then f (x) = f (x1 v1 + · · · + xn vn ) = x1 f (v1 ) + · · · + xn f (vn ) m m X X = x1 ai1 wi + · · · + xn ain wi i=1

= = =

n X

xj

i=1 m X

aij wi

j=1 i=1 m n XX

aij xj wi

i=1 j=1 m X

yi wi

i=1

where in the last line we define yi = have

Pn

j=1

aij xj for i = 1, . . . , m. In matrix form we

y = Ax. In fact y ∈ Rm is the coordinates of f (x) relative to the basis BW . In the above matrix form x and y are written in column matrix form with their coordinates as entries, that is,     x1 y1      x2   y2  ,  . . x= y = . .  .  .    .  xn ym You should verify that the dimensions of the matrices are compatible in the multiplication. It is not difficult to see that if A and B are the matrix representations of f and g with respect to the same bases, then A + B and αA are the respective matrix representations of f +g and αf . For the composite transformation g ◦f in exercise 18, the matrix representation is BA. Example 4.14. Suppose that f : R2 → R2 is a linear transformation that rotates a vector by an angle θ in the counter-clockwise direction. Let B = (e1 , e2 ) be the standard basis in R2 . Figure 4.2 depicts the actions of f on e1 and e2 . The coordinates of their images are (cos θ, sin θ) and (− sin θ, cos θ) respectively. Therefore the 88

4.3. LINEAR TRANSFORMATIONS x2

x2 e2

f (e2 )

f (e1 )

θ

θ

e1

x1

x1

(a) Action on e1

(b) Action on e2

Figure 4.2: The Rotation Transformation matrix representation of f relative to B is A=

4.3.3

! cos θ − sin θ . sin θ cos θ

Linear Functionals

When the co-domain of f is R, the transformation becomes a linear functional. Linear functionals appear frequently in economic models. For example, in production theory, the profit of a firm with net output vector y = (y1 , . . . , yn ) ∈ Rn is a linear functional f (y) = p1 y1 + · · · + pn yn , where pi is the market price of good i, i = 1, . . . , n. The following are some other important examples. Example 4.15. Let A be an n × n square matrix. The trace of A is defined as tr(A) = a11 + a22 + · · · + ann . Example 4.16. The mathematical expectation of a random variable X with a probability density function f (x) is defined as Z



E[X] =

xf (x) dx. −∞

89

CHAPTER 4. LINEAR ALGEBRA Example 4.17. Let [a, b] be an interval in R and let C ([a, b]) be the set of all continuous functions on [a, b]. Then for every g ∈ C ([a, b]), the integral of g, Z

b

g(t) dt

L(g) = a

defines a linear functional L on C ([a, b]).

If V has a finite dimension n, the matrix representation of a linear functional f relative to an ordered basis BV in V is A = [a1 a2 · · · an ]

(4.7)

and f (x) = a1 x1 + a2 x2 + · · · + an xn , where (x1 , . . . , xn ) are the coordinates of x relative to BV . The collection of all linear functionals on V , L(V, R) is a vector space. This vector space V ∗ is often called the dual space of V . It is clear from (4.7) that dim V ∗ = dim V = n. By definition the rank of a linear functional is 1. Therefore by the Dimension Theorem nullity f = dim V − rank f = n − 1. The kernel of f is called a hyperspace of V . Similarly, a hyperplane of f at the level c is defined as1 Hf (c) = {x ∈ V : f (x) = c}. (4.8) For example, Hf (π) = {y ∈ Rn : p1 y1 + · · · + pn yn = π}

(4.9)

is an isoprofit hyperplane in production analysis. The set of y in Hf (π) gives the same profit level π. Similarly, Hf (M ) = {x ∈ Rn : p1 x1 + · · · + pn xn = M } defines the budget constraint in consumer analysis with prices p = (p1 , . . . , pn ) and total expenditure M . Notice that Hf (c) is not a subspace of V unless c = 0. 1

Recall that in functional analysis a hyperplane is the contour of f at c.

90

4.4. INNER PRODUCT SPACES

4.3.4

Multilinear Transformations

Let V be a vector space. Then a function f which maps the product space V r = V × V × · · · × V to a vector space W is multilinear if f (x1 , x2 , . . . , xr ) is a linear in each of the xi when the other xj ’s are held fixed. That is, for all xi , yi ∈ V , α ∈ R, and i = 1, . . . , r, f (x1 , . . . , αxi + yi , . . . , xr ) = αf (x1 , . . . , xi , . . . , xr ) + f (x1 , . . . , yi , . . . , xr ). The function f above is often called r-linear. When r = 2, f is called bilinear. For example, if dim V = n, the matrix representation of a bilinear functional can be expressed as f (x, y) = xT Ay, where A is an n × n square matrix.

4.4

Inner Product Spaces

Let V be a vector space. An inner product on V is a function which assigns an ordered pair of vectors x, y ∈ V a real number, denoted by xT y, which satisfies the following axioms: For all x, y, z ∈ V and α ∈ R, 1. (x + y)T z = xT z + yT z, 2. (αx)T y = α(xT y), 3. xT y = yT x, 4. xT x ≥ 0, xT x = 0 implies that x = 0. Inner products are often denoted by x · y, hx, yi, or (x|y). Example 4.18. When V = Rn , the Euclidean space of dimension n, the dot product of two vector is defined as x T y = x1 y 1 + · · · + xn y n , Example 4.19. In R2 , define xT y as xT y = x1 y1 − x2 y1 − x1 y2 + 4x2 y2 . 91

CHAPTER 4. LINEAR ALGEBRA It is straight forward to show that the above is an inner product. Example 4.20. Let C ([0, 1]) be the set of all continuous functions on [0, 1]. For any f, g ∈ C ([0, 1]), an inner product can be defined as T

Z

1

f (t)g(t)dt.

f g= 0

The study of inner products is concerned with concepts of the “length” of a vector and the “angle” between two vectors. The first idea involves the definition of the norm of a vector as follows: For all x ∈ V , kxk = (xT x)1/2 . The above definition of course satisfies the three axioms of a normed vector space, that is, for all x, y ∈ V and α ∈ R, kxk ≥ 0, k0k = 0,

(4.10)

kαxk = |α| kxk,

(4.11)

kx + yk ≤ kxk + kyk.

(4.12)

Axioms (4.10) and (4.11) are straight forward to verify. Axiom (4.12) requires the application of the Cauchy-Schwarz Inequality and will be presented in section 4.5 below. The following definitions are concerned with vectors being “perpendicular” to each other. Let x, y ∈ V . The angle θ between x and y is implicitly defined as cos θ =

xT y . kxkkyk

It follows that θ is π/2 or 90◦ when the inner product of x and y is zero. Formally, x and y are orthogonal to each other, sometimes written as x ⊥ y, if xT y = 0. A set of vectors S ⊆ V is called an orthogonal set if all pairs of distinct vectors in S are orthogonal. Moreover, if every vector in S has norm equals to 1, S is called an orthonormal set. The orthogonal complement of any set S ⊆ V is defined as all the vectors in V which are orthogonal to every vector in S, that is, S ⊥ = {x ∈ V : xT y = 0, y ∈ S}. 92

4.5. INEQUALITIES

Figure 4.3: An Orthogonal Projection from R3 onto R2 An n × n square matrix A is called orthogonal if the rows of A form a orthonormal set in Rn . Orthogonal matrices appear frequently in econometric analysis. The orthogonal projection of a set S ⊆ V onto a subspace L is a linear transformation f : S → L such that for every x ∈ S, (x − f (x)) ∈ L⊥ . Figure 4.3 illustrates the orthogonal projection of a set S in R3 onto the subspace of the x-y plane. The range of the projection, f (S), is the “shadow” of S on the x-y plane. For any x ∈ S, the vector x − f (x) is parallel to the z axis and therefore orthogonal to the x-y plane.

4.5 4.5.1

Inequalities Cauchy-Schwarz Inequality

Let V be an inner product space. For all x ∈ V , define the norm as kxk = (xT x)1/2 . Then for all x, y ∈ V , |xT y| ≤ kxk kyk.

Proof. The result is trivial if y = 0. So assume that y 6= 0. Then for any α ∈ R, we 93

CHAPTER 4. LINEAR ALGEBRA have 0 ≤ kx − αyk2 = (x − αy)T (x − αy) = xT (x − αy) − αyT (x − αy) = xT x − αxT y − αyT x + α2 yT y = xT x − 2αxT y + α2 yT y. Setting α = xT y/yT y, the inequality becomes 0 ≤ xT x −

(xT y)2 (xT y)2 2 = kxk − , yT y kyk2

and the result follows. Applying the Cauchy-Schwarz inequality to example 4.20 above, we obtain Z

0

1

Z f (t)g(t)dt ≤

1

1/2 Z |f (t)| dt 2

1

1/2 , |g(t)| dt 2

0

0

which is a special case of an important result in functional analysis called H¨older’s Inequality.

4.5.2

Triangular Inequality

For all x, y in a normed vector space V , kx + yk ≤ kxk + kyk. Proof. For x, y ∈ V , we have kx + yk2 = = = ≤ ≤ =

(x + y)T (x + y) xT x + xT y + y T x + y T y kxk2 + 2xT y + kyk2 kxk2 + 2|xT y| + kyk2 kxk2 + 2kxk kyk + kyk2 (kxk + kyk)2

and the result follows. Note that in the second inequality above we have applied the 94

4.6. DETERMINANTS Cauchy-Schwarz Inequality.

4.6

Determinants

Let L(V ) be the collection of all linear operators on a finite-dimensional vector space V . Our goal is to find a necessary and sufficient condition for any f in L(V ) to have an inverse operator. That is, we want to know under what condition f is a bijection. It turns out that a functional on L(V ) can do the trick. The functional, however, is more conveniently defined using the matrix representations of L(V ) relative to a basis BV . Therefore we define a functional D which assigns each n × n matrix A a real number and satisfies the following properties: 1. D is multilinear (recall section 4.3.4) in the row vectors of A. That is, if we let xi be the i-th row of A, i = 1, . . . , n so that we can write D(A) = D(x1 , . . . , xn ), then D(x1 , . . . , αxi + yi , . . . , xn ) = αD(x1 , . . . , xi , . . . , xn ) + D(x1 , . . . , yi , . . . , xn ). 2. If two rows of A are identical, then D(A) = 0. 3. D(I) = 1. That is, the value of the identity matrix is 1. Any functional on L(V ) which satisfies the above three conditions is called a determinant function. It turns out that the determinant function is unique.2 The determinant of A is often written as det A or |A|. The formula for the determinant of A can be expressed inductively as follows. The determinant of a 1 × 1 matrix A = (a) is simply a. For n > 1, define the minor, Aij , of the element aij of A as the determinant of the (n − 1) × (n − 1) matrix by deleting the i-th row and j-th column of A. The determinant of A is given by |A| =

n X

(−1)i+j aij Aij .

j=1 2

See Hoffman and Kunze (1971, Chapter 5).

95

(4.13)

CHAPTER 4. LINEAR ALGEBRA Notice that in (4.13) the row i is not specified. That means we can find the determinant by choosing any row. In fact we can interchange the indices i and j and evaluate |A| along any column instead. Some important properties of the determinant function are given below: 1. |AB| = |A||B|. 2. |AT | = |A|. 3. If A has a row (column) of zeros, then |A| = 0. 4. If B is obtained from A by interchanging two rows (columns) of A, then |B| = −|A|. 5. If B is obtained from A by adding the multiple of one row (column) of A to another row (column), then |B| = |A|. 6. If B is obtained from A by multiplying a row (column) of A by a scalar α, then |B| = α|A|. 7. Suppose A=

A11 A12 0 A22

!

where A11 and A22 are square matrices and 0 represents a zero matrix with conforming dimension. Then |A| = |A11 | |A22 |.

4.7

Inverse Transformations

4.7.1

Systems of Linear Equations

Let f be a linear operator on an n-dimensional vector space V . For a given vector b ∈ V , we are interested in the pre-image f −1 (b). In particular, we want to know if we can find a unique x ∈ V such that f (x) = b. In matrix form we have Ax = b 96

(4.14)

4.7. INVERSE TRANSFORMATIONS where

   A=  

a11 a12 · · · a21 a22 · · · .. .. . . an1 an2 · · ·

a1n a2n .. . ann

     

is the matrix representation of f relative to the standard basis of V , and the coordinates of x and b are written as column vectors:     x1 b1      x2   b2     x =  . , b =  .  .  ..   ..  xn bn Of course if f is a bijection, f −1 is the inverse linear operator such that f −1 ◦ f becomes the identity operator. In matrix form the inverse of A is denoted by A−1 and A−1 A = AA−1 = I (4.15) where I is the n-dimensional identity matrix.3 Pre-multiply both sides of (4.14) by A−1 gives x = A−1 b.

4.7.2

Inverse of a Matrix

Recall in equation (4.13) that the minor, Aij , of the element aij of A is defined as the determinant of the (n − 1) × (n − 1) matrix by deleting the i-th row and j-th column of A. The cofactor of aij , Cij , is defined as (−1)i+j Aij . Using the cofactors, |A| in (4.13) can be written as |A| =

n X

aij Cij .

(4.16)

j=1

Now we show that if we replace the row ai1 , ai2 , . . . , ain in (4.16) with another row k in A, then n X akj Cij = 0. (4.17) j=1 3

See exercise 34 at the end of the chapter.

97

CHAPTER 4. LINEAR ALGEBRA To see this, let B be the matrix obtained from A by replacing the i-row with the k-row. Then B has two identical rows and therefore |B| = 0. The minors of the i-th row of B, however, are still the same as those of A. That is, Bij = Aij for j = 1, . . . , n. Now we have n X

n X

akj Cij =

j=1

j=1 n X

=

(−1)i+j akj Aij (−1)i+j bij Bij

j=1

= |B| = 0. Equations (4.16) and (4.17) can be summarized as n X

( akj Cij =

j=1

|A| if i = k, 0 otherwise.

(4.18)

The transpose of the n × n matrix of cofactors of A is called the adjoint of A, or adj A. Then (4.18) can be expressed as A(adj A) = |A|I.

(4.19)

It is clear from (4.19) that if |A| = 6 0, then the inverse of A defined in (4.15) is A−1 =

1 T 1 (adj A) = C . |A| |A|

where C is the n-dimensional square matrix with elements Cij , the cofactor of aij . If |A| = 0, then the linear operator f is not a bijection and the inverse operator does not exist. In this case f and A are called singular. In the simplest case of n = 2, let A=

a b c d

! .

In order for A to have an inverse, we need |A| = ad − bc 6= 0. The minor of a is d, the minor of b is c, etc. Then C11 = (−1)1+1 d = d and C12 = (−1)1+2 c = −c, etc. 98

4.7. INVERSE TRANSFORMATIONS Putting everything together, we have A−1 =

4.7.3

1 ad − bc

d −b −c a

! .

Cramer’s Rule

Instead of solving for the solution of the whole system of linear equations in (4.14), sometimes we need the solution for a single component, xi . In this case we do not have to find A−1 , which is a very time consuming procedure if done manually. The Cramer’s rule provides a short cut to do just that. Suppose we want the solution of xi in (4.14). We proceed as follows. 1. Find the determinant of A. Ensure that |A| = 6 0. 2. Replace the i-th column of A by the column vector b. Call this modified matrix Ai . In other words,   a11 · · · b1 · · · a1n    a21 · · · b2 · · · a2n  Ai =  .. ..   ..  . . .   an1 · · · bn · · · ann 3. Find |Ai |. 4. The solution is xi =

4.7.4

|Ai | . |A|

Properties of Inverses

Some useful properties of the inverses: 1. If A and B are invertible, then (AB)−1 = B −1 A−1 . 2. (A−1 )−1 = A. 3. (AT )−1 = (A−1 )T . 4. |A−1 | = 1/|A|. 99

CHAPTER 4. LINEAR ALGEBRA 5. The inverse of an upper (lower) triangular matrix is upper (lower) triangular. 6. Suppose A=

A11 0 0 A22

!

where A11 and A22 are square matrices and 0 represents a zero matrix with conforming dimension. Then ! −1 A 0 11 A−1 = 0 A−1 22

4.8

Eigenvectors and Eigenvalues of Linear Operators

Let f be a linear operator on an n-dimensional vector space V . The matrix representation of f relative to any basis of V is an n × n square matrix. In particular, we ask whether there exists an ordered basis B = (x1 , . . . , xn ) such that the matrix representation of f , Λ, is diagonal. If such a basis exists, a lot of important properties of f such as the determinant, rank, the kernel, and the definiteness of its quadratic form (see section 4.9) can be read quickly from Λ. This happens if and only if f (xi ) = λi xi for i = 1, . . . , n. This implies that the range of f is spanned by those xi ’s for which λi 6= 0, and the kernel of f is spanned by the remaining xi ’s. In this section we investigate under what conditions B exists and how it can be found.

4.8.1

Definitions

Let f be a linear operator on an n-dimensional vector space V . Then a nonzero vector x ∈ V is an eigenvector of f if f (x) = λx.

(4.20)

The scalar λ is called an eigenvalue or characteristic value of f . Note that if λ = 1, then x is a fixed point of f . The set of all the eigenvalues of f is called the spectrum of f . The largest of the absolute values of the eigenvalues is called the spectral radius of f . Multiplying both sides of (4.20) by α 6= 0, we have αf (x) = α(λx) 100

4.8. EIGENVECTORS AND EIGENVALUES OF LINEAR OPERATORS or f (αx) = λ(αx) so that eigenvectors are unique up to scalar multiplication, that is, if x is an eigenvector of f , αx is also an eigenvector for any α 6= 0. Conventionally we express eigenvectors in their normalized form, that is, we choose α = 1/kxk so that the resulting eigenvector has norm equal to one. Let A be the matrix representation of f relative to the standard basis of V . Then definition (4.20) becomes Ax = λx so that (A − λI)x = 0.

(4.21)

Since x 6= 0, (4.21) implies that the matrix A − λI is singular so that |A − λI| = 0. By expanding the determinant we get the characteristic equation for λ. This equation is a polynomial of degree n in λ. By the Fundamental Theorem in Algebra we know that it has n roots, which can be complex numbers. Therefore a linear operator on a n-dimensional vector space has n eigenvalues, not necessary distinct. Example 4.21. Suppose that a linear operator f on R2 is represented by the matrix ! 4 2 A= 2 1 relative to the standard basis. Then ! 4−λ 2 . 2 1−λ

(A − λI) = The characteristic equation is

λ2 − 5λ = 0. The roots of this quadratic equation are λ1 = 5 and λ2 = 0. For λ1 = 5, (4.21) implies that ! ! −1 2 x1 = 0, 2 −4 x2 so that x1 = 2x2 , or x1 =

x1 x2

!

! 2 =α . 1

101

CHAPTER 4. LINEAR ALGEBRA √ √ To normalize x1 we let α = 1/ 22 + 12 = 1/ 5 so that the normalized eigenvector is √ ! 2/ 5 √ . x1 = 1/ 5 Similarly, for λ2 = 0, the normalized eigenvector is √ ! 1/ 5 √ . x2 = −2/ 5 We can observe some properties of f from the results. First, the determinant of A is zero since it has a zero eigenvalue. Second, the rank of f is 1. Third, B = {x1 , x2 } is an orthonormal basis for R2 and ! 1 2 1 P = (x1 x2 ) = √ 5 1 −2 is an orthogonal matrix. Finally, if we form a diagonal matrix Λ with the eigenvalues, that is, ! 5 0 Λ= , 0 0 then it is straight forward to show that A = P ΛP T .

4.8.2

Symmetric Operators

A linear operator f is symmetric if for all x, y ∈ V , f (x)T y = xT f (y). In the matrix representation, this means that xT Ay = yT Ax. Since xT Ay = yT AT x by transposition, symmetry requires that A = AT . The eigenvalues and eigenvectors of a symmetric linear operator has many properties that are useful in econometrics and economics: 1. All the eigenvalues are real numbers. 2. If an eigenvalue has k multiple roots, it also has k orthogonal eigenvectors. 3. The n normalized eigenvectors form an orthogonal matrix. For example, if x1 , x2 , . . . , xn are the n normalized eigenvectors of A, then the square matrix 102

4.9. QUADRATIC FORMS P = (x1 x2 · · · xn ) is orthogonal. That is, we have P T P = I, or equivalently, P T = P −1 . 4. Diagonal decomposition (Spectral Theorem): A = P ΛP T where Λ = diag(λ1 , λ2 , . . . , λn ), that is, Λ is an n × n diagonal matrix with the λi at the principal diagonal and the off-diagonal elements are all zero. 5. The sum of the eigenvalues of A (including any multiplicity) is equal to the the sum of the diagonal elements (trace) of A, that is, n X

λi =

i=1

n X

aii .

i=1

6. The product of the eigenvalues of A is equal to its determinant, that is, n Y

λi = |A|.

i=1

7. The rank of A (rank f ) is equal to the number of nonzero eigenvalues. 8. The eigenvalues of A2 (f ◦ f ) are λ21 , λ22 , . . . , λ2n , but the eigenvectors are the same as A. −1 −1 9. If A is nonsingular, the eigenvalues of A−1 are λ−1 1 , λ2 , . . . , λn , but the eigenvectors are the same as A.

10. A matrix A is idempotent if A2 = A. The eigenvalues of an idempotent matrix are either 0 or 1.

4.9

Quadratic Forms

Suppose that f is a symmetric linear operator on a n-dimensional vector space V . A quadratic form is a functional Q : V → R such that T

T

Q(x) = x f (x) = x Ax =

n X n X i=1 j=1

103

aij xi xj ,

CHAPTER 4. LINEAR ALGEBRA where A is the symmetric matrix representation of f .4 The quadratic form is said to process some definite properties if it satisfies one of the following definitions: For all x 6= 0, 1. A is positive definite if xT Ax > 0. 2. A is positive semi-definite if xT Ax ≥ 0, with equality for some x. 3. A is negative definite if xT Ax < 0. 4. A is negative semi-definite if xT Ax ≤ 0, with equality for some x. The quadratic form is called indefinite if it does not satisfy any of the above properties.

4.9.1

Positive Definite Matrices

The following result show two characterizations of positive definiteness. But first we define the principal matrix Ak of A, for k = 1, 2, . . . , n as the k × k submatrix formed by the first k rows and k columns of A. The determinant of Ak , |Ak |, is called the k-th principal minor of A. Theorem 4.4. The following three statements are equivalent: 1. A is positive definite. 2. All eigenvalues of A are positive. 3. All principal minors of A are positive, that is, |Ak | > 0 for k = 1, 2, . . . , n. Statements 2 and 3 of theorem 4.4 imply that a positive definite matrix is nonsingular. Testing for positive semi-definiteness is more involving. First let K = {1, 2, . . . , k} and Π be the set of all permutation on K. For example, for k = 3, Π consists 3! = 3 × 2 × 1 = 6 permutations: π1 = {1, 2, 3}, π2 = {1, 3, 2}, π3 = {2, 1, 3}, π4 = {2, 3, 1}, π5 = {3, 1, 2}, and π6 = {3, 2, 1}. Now denote Aπk as the k × k principal matrix of A with the rows and columns positions rearranged according to the permutation π ∈ Π. For example, the first two permutions of A3 are:   a11 a12 a13   Aπ3 1 = a21 a22 a23  a31 a32 a33 4 Any asymmetric metric B can be converted into a symmetric matrix by taking A = 12 (B + B T ). The values of the quadratic form will be the same.

104

4.9. QUADRATIC FORMS and



Aπ3 2

 a11 a13 a12   = a31 a33 a32  . a21 a23 a22

Theorem 4.5. The following three statements are equivalent: 1. A is positive semi-definite. 2. All eigenvalues of A are nonnegative, with at least one equal to zero. 3. |Aπk | ≥ 0 for k = 1, 2, . . . , n and for all π ∈ Π.

4.9.2

Negative Definite Matrices

Testing for negative and negative semi-definite matrices are similar with a slight difference from the above. Theorem 4.6. The following three statements are equivalent: 1. A is negative definite. 2. All eigenvalues of A are negative. 3. (−1)k |Ak | > 0 for k = 1, 2, . . . , n. Theorem 4.7. The following three statements are equivalent: 1. A is negative semi-definite. 2. All eigenvalues of A are nonpositive, with at least one equal to zero. 3. (−1)k |Aπk | ≥ 0 for k = 1, 2, . . . , n and for all π ∈ Π. Notice, however, that the quadratic form Q(x) is negative (semi-)definite if and only if −Q(x) is positive (semi-)definite. Therefore if A is negative (semi-)definite, we can apply theorems 4.4 and 4.5 to −A. The following example illustrates why we need to examine the permutation matrices when checking for semi-definiteness. Suppose that   0 0 0   A =  0 1 0 . 0 0 −1 105

CHAPTER 4. LINEAR ALGEBRA In this case |A1 | = |A2 | = |A| = 0. Therefore A does not satisfy the requirement of a positive or negative definite matrix. It does, however, satisfy the weak inequality of condition 3 in theorem 4.5 and condition 3 in theorem 4.7 without any permutation. We might want to conclude that A is both positive semi-definite and negative semiT definite. This turns out not to be the case because eT 2 Ae2 = 1 > 0 and e3 Ae3 = −1 < 0 so that A is in fact indefinite.5 The indefiniteness of A is best seen by its eigenvalues, which are 0, 1, and −1, which obviously do not satisfy the condition of any definite matrix. On the other hand, the permutation of A can also reveal its indefiniteness. Take π4 = {2, 3, 1}. Then |Aπ1 4 | = 1, 1 0 |Aπ2 4 | = = −1, 0 −1 which fails the tests in theorems 4.5 and 4.7.

4.10

Exercises

1. Verify that the sets and operations in examples 4.1 to 4.5 satisfy the eight axioms of a vector space. 2. Show that the zero vector in a vector space forms a subspace. 3. Show that all the cases in example 4.8 are subspaces. 4. Let Mn×n be the vector space of all n × n square matrices. A symmetric matrix A is a square matrix equal to its transpose, that is, A = AT . In other words, its elements aij = aji for all i, j = 1, . . . , n. Prove that the set of all n × n symmetric matrices is a subspace of Mn×n . 5. Show that S = {(x, 0) : x ∈ R} and T = {(0, y) : y ∈ R} are subspaces of R2 . (a) Is S ∪ T a subspace of R2 ? (b) Is S + T a subspace of R2 ? 6. Suppose that A is a nonempty subset of a vector space V . Show that the linear span of A is the intersection of all subspaces of V which contain A. 5 The vector ei , i = 1, . . . , n is the i-th standard basis of Rn . Therefore e2 = (0, 1, 0) in this example.

106

4.10. EXERCISES 7. Determine whether each of the following sets is a subspace of the vector space R3 . Justify your answers. (a) B = {(x1 , x2 , x3 ) : x21 + x22 + x23 < 1}, (b) R2 , (c) R3+ , (d) a straight line passing through the origin. 8. Suppose that S and T are subspaces of a vector space V . Show that S ∩ T is a subspace of V . 9. Prove or disprove: Suppose that S and T are subspaces of a vector space V . Then S ∪ T is a subspace of V . 10. Prove or disprove: Suppose that S and T are subspaces of a vector space V . Then S + T is a subspace of V . 11. Consider the vector space R3 . (a) Show that the set of vectors B = {(2, 1, 0), (0, 2, 1), (0, 0, 1)} forms a basis for R3 . (b) Let x = (1, 2, 3). Find the coordinates of x relative to the basis B. 12. Suppose that V = M2×2 is the vector space of all 2 × 2 square matrices. Let ! 1 2 A= . 3 4 What is the coordinates of A relative to the standard basis of M2×2 ? 13. Let f : V → W be a linear transformation. (a) Prove that f (0) = 0. (b) Show that the kernel of f is a subspace of V . (c) Show that the range of f , f (V ), is a subspace of W . 107

CHAPTER 4. LINEAR ALGEBRA 14. Suppose that the linear operator f on the vector space R3 is defined by f (x, y, z) = (z, 0, y). (a) Find the kernel of f . (b) Use the Dimension Theorem to calculate the rank of f . (c) Find the matrix representation of f relative to the standard basis. 15. Let f : V → W be a linear transformation. Prove or disprove: The graph of f is a subspace of the product vector space V × W . 16. Let V , W , and Z be vector spaces and let f : V → W and g : W → Z be linear transformations. Show that g ◦ f is also a linear transformation. 17. Suppose that a linear operator f on R3  0  A = 0 0

is represented by the matrix  0 0  1 0 , 0 −1

relative to the standard basis. (a) Identify the kernel of f . (b) What is the nullity of f ? (c) What is the rank of f ? (d) Do your results above conform with the Dimension Theorem? Explain. 18. Let the linear functions f : R3 matrices  3 −2  1 −1 5 5

→ R3 and g : R3 → R2 be represented by the  0  4 3

and

2 −3 4 5 1 2

!

respectively. Find the matrix representation of the composite function g ◦ f . 19. Show that dim L(V, W ) = dim V × dim W . 20. Show that each of the examples in section 4.3.3 is a linear functional. 21. Verify that each of the examples in section 4.4 is an inner product space. 108

4.10. EXERCISES 22. Define a function f : R2 → R by f (x, y) = |x| + |y|. Is (R, f ) an inner product space? Explain. 23. Show that for every x ∈ V , 0T x = 0. 24. Let V be a normed vector space. Show that (V, ρ) is a metric space, where for every x, y ∈ V , ρ(x, y) = kx − yk. 25. Let V be a normed vector space. Prove the parallelogram equality: For every x, y ∈ V , kx + yk2 + kx − yk2 = 2(kxk2 + kyk2 ). 26. Consider the relation ⊥ on a vector space V . Determine if the relation is (a) reflexive, (b) transitive, (c) circular, (d) symmetric, (e) asymmetric, (f) antisymmetric. 27. Suppose that (2x + y) ⊥ (x − 3y) and kxk = 2kyk. Find the angle between x and y. 28. Suppose that S is a one-dimensional subspace of a vector space V . Let {b} be a basis of S. (a) Define S ⊥ , the orthogonal complement of S. (b) Show that S ⊥ is a subspace of V . 29. Let S be a subspace of a vector space V . Show that S ⊥ is also a subspace of V . 30. Give a geometric interpretation of the dot product in R3 . What if one of the vector is a unit vector? 109

CHAPTER 4. LINEAR ALGEBRA 31. Suppose that L(V ) is the set of all linear operators on an n-dimensional vector space V . Let D : L(V ) → R be the determinant function. Prove or disprove: D is a linear functional. 32. Prove statements 2, 3, 5, and 6 in section 4.6. 33. Let A, B, and C be the matrix representations of three linear operations on a vector space. The three matrices are identical except for their first rows, which are a1 , b1 and αa1 + βb1 respectively, where α, β ∈ R. Show that |C| = α|A| + β|B|. 34. Show that if BA = AC = I, then B = C. 35. Let the matrix representation relative to the standard basis of the linear operator f : R2 → R2 be ! 2 3 A= . 3 4 (a) Show that f is invertible. (b) Show that f −1 is represented by the matrix !

A−1 =

−4 3 . 3 −2

(c) Is it true that |A| = 1/|A−1 |? 36. Let

  2 0 −1   A = 5 1 0  , 0 1 3



 3 −1 1   B = −15 6 −5 . 5 −2 2

Show that A is the inverse of B. 37. Find the inverses of the following matrices: (a)

cos θ sin θ − sin θ cos θ

!

110

4.10. EXERCISES 

 −3 6 −11   (b)  3 −4 6  4 −8 13 38. Suppose that f and g are two invertible linear operators on a vector space V . Prove that (g ◦ f )−1 (x) = (f −1 ◦ g −1 )(x) for all x ∈ V . 39. Find the inverse of



 1 1 0   B = 0 1 0 . 0 0 1

If B is the matrix representation of a linear operator f , find (a) the kernel of f , (b) the rank of f . 40. Consider the following system of linear equations: x1 − 2x2 = −8, 5x1 + 3x2 = −1. (a) Express the system in matrix form. (b) Show that the solution exists. (c) Solve x1 and x2 by a matrix inversion. 41. Given the system of linear equation 2x1 − x2 + 2x3 = 2 x1 + 10x2 − 3x3 = 5 −x1 + x2 + x3 = −3.

(a) Express the system in matrix form. (b) Find the solution for x3 . 111

CHAPTER 4. LINEAR ALGEBRA 42. Prove that if A and B are invertible, then (AB)−1 = B −1 A−1 . 43. Prove that if A is invertible, then (A−1 )−1 = A. 44. Find the eigenvalues and normalized eigenvectors of the following matrices: ! ! ! ! 3 4 1 1 −5 2 1 2 (i) A = , (ii) A = , (iii) A = , (iv) A = . 4 −3 4 1 2 −2 2 1 Show that

Pn

i=1

λi =

Pn

i=1

aii , P T P = I, A = P ΛP T , and |Λ| = |A|.

45. Let the matrix representation of a linear operator be ! 3 6 A= . 1 4 (a) Find the eigenvalues of A. (b) Find the normalized eigenvectors of A. (c) Are the eigenvectors orthogonal to each other? Why or why not? 46. Suppose that the linear operator f on  1  0 0

R3 is represented by the matrix  0 0  3 2 2 0

relative to the standard basis. (a) Find the eigenvalues and the normalized eigenvectors. (b) Determine the definiteness of the quadratic form. 47. Prove statement 1 in section 4.8.2. Hint: Suppose that A has a complex eigenvalue √ λ + iβ where i = −1 with the corresponding eigenvector x + iy. Then A(x + iy) = (λ + iβ)(x + iy). Show that β = 0. 48. Prove that for distinct eigenvalues λ1 and λ2 of a symmetric operator A, the corresponding eigenvectors x1 and x2 are orthogonal. 112

4.10. EXERCISES 49. Prove the Spectral Theorem in section 4.8.2. Hint: Recall that columns in P are pairwise orthonormal, that is, ( 1, if i = j, xT i xj = 0, otherwise. 50. Prove statement 6 in section 4.8.2. Hint: first show that |P | = ±1 and use the Spectral Theorem. 51. Prove statements 8, 9, and 10 in section 4.8.2. 52. Let X be an n × k matrix where k ≤ n and the columns of X are linearly independent so that (X T X)−1 exists. (a) Show that M1 = X(X T X)−1 X T and M2 = In − X(X T X)−1 X T are idempotent matrices. (b) Suppose that the diagonal decomposition of M1 = P Λ1 P T . Show that M2 = P Λ2 P T where Λ1 + Λ2 = In . 53. Let A be an n × n matrix (not necessary symmetric) which has nonnegative P elements and satisfies nj=1 aij = 1 for i = 1, . . . , n. Show that AT has at least one unit eigenvalue. Hint: Consider the simplex ( S=

x ∈ Rn : xi ≥ 0,

n X

) xi = 1 .

i=1

Show that the linear function f represented by AT is a linear operator on S, that is, for every x ∈ S, f (x) is also in S. Then show that f : S → S satisfies the hypothesis of the Brouwer Fixed Point Theorem. A is called a transition matrix and is important in the study of Markov process in macroeconomics (see Ljungqvist and Sargent, 2004, 30–31). 113

CHAPTER 4. LINEAR ALGEBRA 54. Let



 0 0 0   A = 0 −3 0 , 0 0 2

  1 0 0   B = 0 3 0 , 0 0 0



 −3 0 0   C =  0 −1 0  . 0 0 −2

(a) Determine the definiteness of A, B, and C. (b) Determine the definiteness of A + B + C. 55. Let X be an n × k matrix where k ≤ n and the columns of X are linearly independent. Show that X T X is positive definite. 56. Let A be a positive definite matrix. Show that A = QQT where Q is an invertible matrix. 57. Let A be a positive definite matrix. Prove that A−1 is also positive definite. 58. Suppose that f is a symmetric linear operator on a n-dimensional vector space V . Prove the polarization identity: For every x, y ∈ V , 1 1 xT f (y) = Q(x + y) − Q(x − y). 4 4

114

Chapter 5 Vector Calculus In this chapter we introduce multivariate differential calculus, which is an important tool in economic modelling. Concepts developed in previous chapters are applied to the Euclidean space, which is both a metric space and a normed vector space.

5.1 5.1.1

Vector Differentiation Introduction

Recall that in single variable calculus, the derivative of a function f : R → R at a point x is defined by f (x + h) − f (x) , f 0 (x) = lim h→0 h provided that the limit exists. The limit can be expressed as f (x + h) − f (x) − f 0 (x)h = 0. h→0 h lim

(5.1)

If the limit exists at every point in a set E ⊆ R, we say that f is differentiable on E. Differentiability implies that the function is continuous on E, but the converse is not true. There are many examples of continuous functions which are not differentiable. 115

CHAPTER 5. VECTOR CALCULUS Example 5.1. Let f (x) = x2 . The the limit in (5.1) is (x + h)2 − x2 − f 0 (x)h x2 + 2hx + h2 − x2 − f 0 (x)h = lim h→0 h→0 h h 2 0 2hx + h − f (x)h = lim h→0 h = lim (2x + h − f 0 (x)) lim

h→0

= 2x − f 0 (x), which gives f 0 (x) = 2x. For any particular point, say x = −1, the derivative f 0 (−1) = −2. The last term in the numerator of the limit in (5.1) is f 0 (x)h = −2h, which is a linear function in h. Geometrically, f 0 (x) is the slope of the function f (x) at the point x. Now we extend the concept of differentiation to the case of functions of several variables. Recall from chapters 3 and 4 that the Euclidean space Rn is both a a metric space and a normed vector space, with the metric defined as ρ(x, y) = kx − yk, for every x, y ∈ Rn . Suppose that S is an open set in Rn , f maps S into Rm , and x ∈ S. If there exists a linear transformation, Df (x) : Rn → Rm such that, for h ∈ Rn , kf (x + h) − f (x) − Df (x)(h)k lim = 0, (5.2) h→0 khk then f is differentiable at x and Df (x) is called the derivative of f at x. If f is differentiable at every point in S, we say that f is differentiable on S. Since Df (x) is linear, it can be represented by a m × n matrix relative to the standard bases in Rm and Rn . If m = n, Df (x) becomes a linear operator. In this case, the determinant of Df (x) is called the Jacobian, Jf (x), of f at x. Example 5.2. Suppose that f : Rn → Rm is a linear function. Then f can be represented by a m×n matrix A relative to the standard bases such that f (x) = Ax. The limit in definition (5.2) becomes kA(x + h) − Ax − Df (x)(h)k kAx + Ah − Ax − Df (x)(h)k = lim h→0 h→0 khk khk kAh − Df (x)(h)k = lim . h→0 khk lim

116

5.1. VECTOR DIFFERENTIATION It is clear that the limit is zero if Df (x) = A for all x ∈ Rn . If m = n, then Df (x) is a square matrix and the Jacobian is Jf (x) = |A|, the determinant of A. For m = n = 1, the function reduces to the single variable case that f (x) = ax so that f 0 (x) = a. Some important properties of Df (x) are listed below: 1. If f is differentiable, then Df (x) is unique.1 2. f (x) + Df (x)h is a linear approximation of f (x + h). 3. A function f is continuous if it is differentiable. 4. Let F (Rn , Rm ) be the collection of all differentiable functions from Rn to Rm . The derivatives are linear functions on F (Rn , Rm ), that is, if f, g ∈ F (Rn , Rm ), then for any real number α, D(αf + g)(x) = αDf (x) + Dg(x). 5. Chain rule: Suppose that f : Rn → Rm and g : Rm → Rl are differentiable. Then D(g ◦ f )(x) is a linear function from Rn into Rl such that D(g ◦ f )(x) = Dg(f (x))Df (x). 6. Product rule: Suppose that f and g are functionals on S and define f g(x) = f (x)g(x). Then D(f g)(x) = f (x)Dg(x) + g(x)Df (x). Next we define the partial derivatives of f . Suppose that f maps an open set S ⊆ Rn into Rm . Let (e1 , e2 , . . . , en ) and (u1 , u2 , . . . , um ) be the standard bases for Rn and Rm respectively. Then f can be expressed as its component functionals relative to (u1 , u2 , . . . , um ):   f1 (x)    f2 (x)   f (x) =   ..  ,  .  fm (x) 1

See Rudin (1976, p. 213) for a proof.

117

CHAPTER 5. VECTOR CALCULUS where fi (x) = f (x)T ui , for i = 1, . . . , m. The partial derivative of fi with respect to xj is defined as ∂fi (x) fi (x + tej ) − fi (x) = lim , t→0 ∂xj t provided that the limit exists. The following theorem reveals the matrix representation of the linear function Df (x) relative to the standard bases. Theorem 5.1. Suppose that f defined above is differentiable at a point x ∈ S. Then the partial derivatives exist and Df (x)ej =

m X ∂fi (x) i=1

∂xj

ui ,

j = 1, . . . , n.

See Rudin (1976, p. 215–6) for a proof of this theorem. Consequently, the matrix representation of Df (x) relative to the above standard bases is 

∂f1 (x)/∂x1   ∂f2 (x)/∂x1  ..  . 

∂f1 (x)/∂x2 ∂f2 (x)/∂x2 .. .

 ∂f1 (x)/∂xn  ∂f2 (x)/∂xn  . ..  . 

··· ···

∂fm (x)/∂x1 ∂fm (x)/∂x2 · · ·

5.1.2

∂fm (x)/∂xn

Functional Analysis

In the case of m = 1, Df (x) is a linear functional and is often called the gradient of f at x and is denoted by ∇f (x). Although Df (x) is a 1 × n matrix, we usually write ∇f (x) as a column vector like x. Therefore in partial derivative form we have 

 ∂f (x)/∂x1    ∂f (x)/∂x2  . ∇f (x) =  ..   .   ∂f (x)/∂xn

Example 5.3. Supose f (x1 , x2 ) = axα1 xβ2 , then ∇f (x) =

∂f (x)/∂x1 ∂f (x)/∂x2

! =

αaxα−1 xβ2 1 βaxα1 xβ−1 2

! .

Notice that ∇f (x) is a linear functional for any particular value of x. For example, 118

5.1. VECTOR DIFFERENTIATION

Table 5.1: Some Common Functionals Example f (x) ∇f (x) ∇2 f (x) T 1 y x y 0n×n T 2 x Ay Ay 0n×n T 3 x x 2x 2In 4 yT xxT y 2yyT x 2yyT 5 xT Ax (A + AT )x A + AT when x = (1, 1), ∇f (1, 1) =

αa(1)α−1 (1)β βa(1)α (1)β−1

! =

! αa . βa

In the above example, a different x of course gives a different functional. We can, however, view the gradient as a (non)linear function which maps vectors in S into vectors in Rn . In other words, we can treat the matrix representation of the gradients as vectors in Rn . Then we can replace f with ∇f (x) in (5.2) and the resulting linear operator, if it exists, is called the second derivative or Hessian of f at x, ∇2 f (x). We say that f is twice differentiable at x. The matrix representation of the Hessian is by definition a n × n square matrix in terms of the partial derivatives, that is,   ∂ 2 f (x)/∂x1 ∂x1 ∂ 2 f (x)/∂x1 ∂x2 · · · ∂ 2 f (x)/∂x1 ∂xn  2   ∂ f (x)/∂x2 ∂x1 ∂ 2 f (x)/∂x2 ∂x2 · · · ∂ 2 f (x)/∂x2 ∂xn  2 . ∇ f (x) =  .. .. ..   . . .   ∂ 2 f (x)/∂xn ∂x1 ∂ 2 f (x)/∂xn ∂x2 · · · ∂ 2 f (x)/∂xn ∂xn If f is twice differentiable at x and each element of the Hessian is a continuous function at x, f is called a C 2 function. Also, the Schwarz Identity (often called Young’s Theorem) states that the Hessian is symmetric, that is, for i 6= j, ∂ 2 f (x) ∂ 2 f (x) = . ∂xi ∂xj ∂xj ∂xi Table 5.1 lists the gradients and Hessians of some commonly used functionals. The square matrix yyT in example 4 is called the outer product of vector y. In example 5, if A is symmetric, the gradient and Hessian of xT Ax are 2Ax and 2A respectively. Also, if A is the identity matrix, the results reduce to those in example 3. Geometrically, the partial derivative of a functional f with respect to the standard 119

CHAPTER 5. VECTOR CALCULUS

Figure 5.1: Gradient of a Functional basis, ∂f (x)/∂xj , is the rate of increase of f (x) at x in the direction of the j-th coordinate. The definition can be modified to find the rate of increase in an arbitrary direction. Suppose u ∈ Rn is a unit vector, with kuk = 1. Then the directional derivative of f at x in the direction of u is defined as f (x + tu) − f (x) . t→0 t

Du f (x) = lim

It can be shown that if f is differentiable,2 Du f (x) = ∇f (x)T u. It is clear that Du f (x) achieves its maximum value when u has the same direction as ∇f (x). In other words, the gradient points to the direction in which the value of f is increasing at the fastest rate, with the rate of increase given by k∇f (x)k. Figure 5.1 illustrates the relationship between the gradient and the graph of f in the case of n = 2. Notice that ∇f (x0 , y0 ) is orthogonal to the tangent line to the level curve at (x0 , y0 ). Theorem 5.2 (Mean Value Theorem). Suppose that f is a differentiable functional 2

See exercise 6 at the end of the chapter.

120

5.2. APPROXIMATION OF FUNCTIONS on a convex set S ⊆ Rn . For all x, y ∈ S, there exists a point z = αx + (1 − α)y for some α ∈ [0, 1] such that f (x) − f (y) = ∇f (z)T (x − y).

5.2

Approximation of Functions

Unlike, say, Newton’s law of gravity, the exact functional forms in economic modelling are rarely known. Often we choose a functional form to satisfy certain structural restrictions from the model. For analytical convenience, we also want to have functional forms that are easy to manipulate. In this section we look at approximation of functions. The following theorem provide the starting point. It assures us that any continuous real valued function defined on a closed interval can be approximated uniformly by polynomials. Theorem 5.3 (Stone-Weierstrass Theorem). Let f : [a, b] → R be a continuous function. Then there exists a sequence of polynomials Pn (x) that converges uniformly to f on [a, b]. The proof of this remarkable result can be found in Brosowski and Deutsch (1981). If the function f is differentiable, the polynomial approximation becomes the well-known Taylor series.

5.2.1

Taylor Series

Let f (x) be a single variable function of x which is differentiable to any order s. The Taylor expansion of f centred at a point x∗ is given by f (x) =

∞ X f (s) (x∗ ) s=0

s!

(x − x∗ )s

1 1 = f (x∗ ) + f 0 (x∗ )(x − x∗ ) + f 00 (x∗ )(x − x∗ )2 + f 000 (x∗ )(x − x∗ )3 + · · · 2 3! Figure 5.2 depicts approximations of y = sin x at x∗ = 0. Notice that the approximation becomes better as the order increases. Nevertheless, the approximated values deviate dramatically from the true values once x exceed a certain distance from x∗ . In economic applications we often take the approximation up to the first or second derivative term, which we called first-order and second-order approximations respectively. 121

CHAPTER 5. VECTOR CALCULUS y 5

P3 (x)

4

P1 (x)

3 2 1

sin x

−6 −5 −4 −3 −2 −1 0 −1

P5 (x)

1

2

3

4

5

6

x

−2 −3 −4 −5

P7 (x)

Figure 5.2: Taylor Approximations of y = sin x In the multivariate cases Taylor approximations normally go up to second order only because of the complexity of higher derivatives. Theorem 5.4 (Taylor’s Theorem). Suppose f is a C 2 functional on a convex set S ⊆ Rn . Then for x0 , x ∈ S, 1 f (x) = f (x0 ) + ∇f (x0 )T (x − x0 ) + (x − x0 )T ∇2 f (x0 )(x − x0 ) + r(x0 , x), 2 where limx→x0 r(x0 , x) = 0. Taylor’s theorem provides us a second-order approximation (sometimes called a 2-jet) of an arbitrary C 2 functional about a point x0 by ignoring the remainder term r(x0 , x). In dynamic models x0 is often chosen as the steady-state (long-run) value of x. A functional which can provide a second-order approximation to an arbitrary 2 C functional is called a flexible functional form and is popular among applied econometricians. The quadratic functional form is defined as 1 f (x) = α0 + αT x + xT Ax, 2

(5.3)

where α0 ∈ R, α ∈ Rn and A = AT , an n × n symmetric matrix, are unknown 122

5.2. APPROXIMATION OF FUNCTIONS y x

1

log(1 + x)

−1

x

0

1

−1

Figure 5.3: Logarithmic Approximation parameters to be estimated from the regression model. The total number of unknown parameters is 1+n+n(n+1)/2 and so if n is big the regression needs a large number of observations. Another popular flexible functional form is the translog function, defined as 1 log f (x) = β0 + β T log x + log xT B log x, 2 where log x = (log x1 log x2 · · · log xn )T , and β0 ∈ R, β ∈ Rn and B = B T , an n × n symmetric matrix, are again the unknown parameters.

5.2.2

Logarithmic Approximation

The function f (x) = log(1+x) is tangent to the 45 degree line at x = 0 (See figure 5.3 and exercise 12 below). Therefore for small values of x, a useful approximation is log(1 + x) ' x. For example, let Pt be the price level in period t and let pt = log Pt . Then ∆pt = pt − pt−1 = log Pt − log Pt−1     Pt Pt − Pt−1 = log = log 1 + Pt−1 Pt−1 = log(1 + πt ) ' πt . 123

(5.4)

CHAPTER 5. VECTOR CALCULUS That is, the log difference of price levels between the two periods is approximately equal to the inflation rate πt . Similarly, the log difference of output levels between two periods can be used as the growth rate in practice. On the other hand, for x around 1, the approximation can be modified to become log x ' x − 1.

5.2.3

(5.5)

Log-Linear Approximation

Another useful approximation in economics is the log-linear approximation. Let f (x) be any differentiable function. Using the Taylor first-order approximation at a point x∗ , we have f (x) ' f (x∗ ) + f 0 (x∗ )(x − x∗ ) x  = f (x∗ ) + x∗ f 0 (x∗ ) ∗ − 1 x  x ∗ ∗ 0 ∗ ' f (x ) + x f (x ) log ∗ (using (5.5)) x = f (x∗ ) + x∗ f 0 (x∗ )(log x − log x∗ ). The above result is often written as f (x) ' f (x∗ ) + x∗ f 0 (x∗ )ˆ x where xˆ = log x − log x∗ . In view of (5.4), xˆ can be interpreted as the proportion of x deviated from x∗ . For example, let f (x) = ex . Then the log-linear approximation at x∗ = 1 is ex ' e1 + (1)e1 (log x − log(1)) = e + e log x = e(1 + log x). Figure 5.4 depicts ex and its log-linear approximation at x∗ = 1. See Clausen (2006) for more discussions on the topic. See Wickens (2011, p. 506) for an application of the log-linear approximation in solving a real business cycle model. 124

5.3. INVERSE FUNCTIONS y ex

7 6 5 4 3 2 1 0 −1

e(1 + log x)

1

2

x

Figure 5.4: Log-Linear Approximation

5.3

Inverse Functions

Recall that if f is a linear operator on a vector space V and f is a bijection, then f is invertible. In matrix representation with respect to a basis of V , we have y = Ax and x = A−1 y. Now suppose that f is nonlinear and continuously differentiable (C 1 ) function. Under what conditions can we be sure that f has an inverse in a neighbourhood? It may be very difficult to find an explicit inverse of f . But can we find instead the derivative of f −1 ? The answers to both questions are the results of the inverse function theorem: Theorem 5.5 (Inverse Function Theorem). Suppose f is a C 1 mapping of an open set S ⊆ Rn into Rn . If the Jacobian Jf (x) 6= 0 at some point x ∈ S and y = f (x). Then 1. there exist open sets U and V in Rn such that x ∈ U, y ∈ V , f : U → V is a bijection; 2. the derivative of the inverse function f −1 : V → U is given by Df −1 (y) = [Df (x)]−1 .

(5.6)

In other words, the function f is bijective at each point x ∈ U . An invertible function f : Rn → Rn is often called a diffeomorphism. Example 5.4. Let f : R2 → R2 be given by f (x) = ((x21 + x22 )/2, x1 x2 ). 125

CHAPTER 5. VECTOR CALCULUS The range of f is {(y1 , y2 ) ∈ R2 : y1 ≥ 0}. The derivative of f is x1 x2 x2 x1

Df (x) =

! ,

with Jacobian Jf (x) = x21 − x22 . Therefore f is not invertible when x1 = ±x2 . Put x = (1, 0) and let y = f (x). Then by (5.6) Df −1 (y) = [Df (x)]−1 =

1 0 0 1

!−1 =

1 0 0 1

!

Sometimes we have a system of n nonlinear equations with n endogenous variables x and m exogenous variables θ (often called parameters). Instead of finding an explicit formula for x in terms of θ, we can study the effect of changes in x for changes in θ, which we call comparative statics. The implicit function theorem is an useful tool for doing that. If f (x, θ) is a function of x ∈ Rn and θ ∈ Rm into Rn , then we denote the derivative of f with respect to x by Dx f (x, θ) and similarly θ by Dθ f (x, θ). Theorem 5.6 (Implicit Function Theorem). Let f be a C 1 mapping of an open set S ⊆ Rn+m into Rn such that f (x, θ) = 0 for some point (x, θ) ∈ S. If Dx f (x, θ) is invertible, then there exist open sets U ⊂ Rn+m and W ⊂ Rm , with (x, θ) ∈ U , θ ∈ W , and a C 1 mapping g : W → Rn such that x = g(θ), f (g(φ), φ) = 0

(5.7)

for all φ ∈ W , −1

Dg(θ) = −[Dx f (x, θ)] Dθ f (x, θ).

(5.8) (5.9)

Equation (5.7) means that the solution of the endogenous variable x exists as a function of the exogenous variable θ in the neighbourhood W . Equation (5.8) ensures that the solutions g(φ) for all φ ∈ W satisfy the system of equations f (x, θ) = 0. And finally, although an explicit solution of the function g may be difficult to obtain, we can get the derivative of g by equation (5.9). The elements of the matrix Dg(θ) are the rates of change of the endogenous variables in x with respect to the exogenous variables in θ. Example 5.5. We look at the simple case that f is a linear transformation from Rn+m to Rn . Then f can be represented by a matrix relative to the standard basis. 126

5.3. INVERSE FUNCTIONS We are interested in all the vectors (x, θ) that belong to the kernel of f . That is, " # i x f (x, θ) = A B = 0 ∈ Rn , θ h

where A is an n × n matrix and B is an n × m matrix. This means that Ax + Bθ = 0. If A is invertible, then x = g(θ) = −A−1 Bθ, which is an explicit solution in equation (5.7). For any vector φ ∈ Rm , f (g(φ), φ) = A(−A−1 Bφ) + Bφ = 0, which verifies equation (5.8). Finally, the rate of change of x with respect to θ is Dg(θ) = −A−1 B. This is equation (5.9) with the facts that Dx f (x, θ) = A and Dθ f (x, θ) = B.

Example 5.6. Consider the following Keynesian model for the aggregate economy: Y = C[(1 − t)Y ] + I(r, Y ) + G, M/P = L(Y, r), The first equation is often called the IS equation and the second LM equation. In this model, Y (output) and r (interest rate) are endogenous and P (price), G (government expenditure), t (tax rate), and M (money supply) are exogenous. C, I, and L are functions for consumption, investment, and money demand respectively. In the context of theorem 5.6, we have n = 2, m = 4, with x = [Y r]T and θ = [t G M P ]T . The function f is ! Y − C[(1 − t)Y ] − I(r, Y ) − G f (x, θ) = . M/P − L(Y, r) 127

CHAPTER 5. VECTOR CALCULUS The derivative of f with respect to x is Dx f (x, θ) =

1 − C 0 [·](1 − t) − ∂I/∂Y −∂L/∂Y

! −∂I/∂r . −∂L/∂r

(5.10)

In order for the model to work and to have an implicit solution, Dx f (x, θ) must be invertible, that is, the Jacobian cannot be zero. In this IS-LM model, we normally assume that 1 − C 0 [·](1 − t) − ∂I/∂Y is a positive fraction, ∂I/∂r < 0, ∂L/∂Y > 0, and ∂L/∂r < 0. Therefore the Jacobian is J = −[1 − C 0 [·](1 − t) − ∂I/∂Y ](∂L/∂r) − (∂I/∂r)(∂L/∂Y ) > 0. The derivative of f with respect to θ is Dθ f (x, θ) =

C 0 [·]Y 0

! −1 0 0 . 0 1/P −M/P 2

(5.11)

Using (5.9) in Theorem 5.6 we can obtain Dg(θ) using (5.10) and (5.11), which tells the effects of changes in t, G, M , and P on output Y and interest rate r. If the purpose of the analysis is to study the impact of one exogenous variable on one endogenous variable only, then we can use Cramer’s rule instead of inverting Dx f (x, θ). For example, to find the impact of a tax increase on output, we replace the first column of Dx f (x, θ) with the first column of Dθ f (x, θ), divide the determinant with J, and add a negative sign: 0 1 C [·]Y −∂I/∂r ∂Y = − ∂t J 0 −∂L/∂r C 0 [·]Y (∂L/∂r) = < 0. J

5.4

Exercises

1. Let f be a linear operator on Rn . Find the derivative of f by using the partial derivatives in theorem 5.1. 2. Find the derivative Df (x) of the function f : R2 → R2 , defined by f (x) = (x1 + sin x2 , x2 ex1 ). 128

5.4. EXERCISES 3. Find ∇f (x) and ∇2 f (x) in equation (5.3). 4. Find the Hessian ∇2 f (x) of the function f : R2++ → R at x = (2, 1) such that f (x) = x1 +



x2 .

Determinate the definiteness of ∇2 f (x) at that point. 5. Let f : R2++ → R, f (x) = (x1 x2 )1/2 . (a) Find the gradient ∇f (x) at the point x = (1, 1). (b) What is the directional derivative Du f (x) at (1, 1) in the direction of the √ √ vector u = (2/ 5, 1/ 5)? 6. Suppose that f is a differentiable functional on Rn . Suppose u ∈ Rn is a unit vector, with kuk = 1. Show that the directional derivative of f at x in the direction of u is Du f (x) = ∇f (x)T u. 7. Prove that the gradient of a differentiable functional f points in the direction of greatest increase. Hint: Use the Cauchy-Schwarz Inequality. 8. Suppose that f (x) = x2 − 1. (a) Does f −1 exist in the neighbourhood of x = 0? (b) Find the second order Taylor formula for f about the point x = 0. 9. Calculate the directional derivative of the function f (x, y) = x + 2 log y at the point (1, 2) in the direction (1, 1). 10. Let f : R2 → R be given by f (x, y) =

1 . 1 + x2 + y 2

(a) Calculate the directional derivative of f at the point (1, 0) in the direction of the vector v = (4, 3). (b) In which direction the value of f increases at the fastest rate? Explain. 129

CHAPTER 5. VECTOR CALCULUS 11. Find the second order Taylor formula for the following functions about the given point x0 in R2 . (a) f (x) = (x1 + x2 )2 , x0 = (0, 0) (b) f (x) = x1 x2 + (x1 + x2 )3 , x0 = (0, 0) (c) f (x) = ex1 +x2 , x0 = (0, 0) (d) f (x) = sin(x1 x2 ) cos x2 , x0 = (1, 0) 12. Find the first-order and second-order Taylor approximations of the following functions at the indicated points: (a) f (x) = sin x, at x∗ = 0 (b) f (x) = sin x, at x∗ = π/2 (c) f (x) = ex , at x∗ = 1 (d) f (x) = log(1 + x), at x∗ = 0 13. In a dynamic growth model the relation between consumption in period t + 1, ct+1 , consumption in period t, ct , and the capital stock in period t + 1, kt+1 , is governed by the Euler equation β

U 0 (ct+1 ) 0 [F (kt+1 ) + 1 − δ] = 1, U 0 (ct )

where U is a utility function, F is a production function, and β and δ are given parameters. Let x = [ct+1 ct kt+1 ]T . Show that the first-order Taylor approximation of the Euler equation about the steady-state values x∗ = [c∗ c∗ k ∗ ]T is  U 00 (c∗ ) 0 ∗ 00 ∗ ∗ β F (k ) + 1 − δ + 0 ∗ [F (k ) + 1 − δ](ct+1 − ct ) + F (k )(kt+1 − k ) ' 1. U (c ) 

0



14. Prove the following log-linear approximations: (a) x ' x∗ (ˆ x + 1) (b) xy ' x∗ y ∗ (ˆ x + yˆ + 1) (c) yt = ct + gt can be represented by yˆt =

c∗ g∗ c ˆ + gˆt . t y∗ y∗

130

5.4. EXERCISES 15. Prove the following multivariate version of log-linear approximation:3 Let f : Rn → R be a C 1 function. Then for any x∗ = [x∗1 x∗2 · · · x∗n ]T ∈ Rn , f (x) ' f (x∗ ) + ∇f (x∗ )T Dˆ x, where D is an n × n diagonal matrix with diagonal elements x∗1 , x∗2 , . . . , x∗n and ˆ = [log x1 − log x∗1 log x2 − log x∗2 · · · log xn − log x∗n ]T . x 16. Let f : R2 → R2 be given by f (x) = (x21 − x22 , 2x1 x2 ). (a) What is the range of f ? (b) Is the Jacobian of f zero at any point in R2 ? (c) Put x = (2, 1) and let y = f (x). Compute Df (x) and Df −1 (y). 17. Let f : R2 → R2 be given by f (x) = (x1 cos x2 , x1 sin x2 ). Find the set of points that the inverse of f does not exist. 18. Let f : R2 → R2 be the function f (x) = (ex1 cos x2 , ex1 sin x2 ). (a) Find the derivative Df (x). (b) Evaluate the Jacobian of f at the point a = (0, π/3). (c) Let b = f (a). Find the derivative Df −1 (b). 19. Define the H´enon map f : R2 → R2 by f (x) = (a − bx2 − x21 , x1 ), where a and b are constants and b 6= 0. 3

The idea is due to Shivaji Koirala.

131

CHAPTER 5. VECTOR CALCULUS (a) Show that Df (x) is invertible at any point of R2 . Thus every point of R2 has a neighbourhood in which f is a bijection. (b) Find an explicit formula for f −1 . Derive Df −1 (x), and verify the inverse function theorem. The H´enon map exhibits a number of interesting properties in the study of complex dynamics. See Devaney (2003) for details. 20. Show that the equations xy 5 + yu5 + zv 5 = 1 x5 y + y 5 u + z 5 v = 1 have a unique solution u = g1 (x, y, z), v = g2 (x, y, z) in the neighbourhood of the point (x, y, z, u, v) = (0, 1, 1, 1, 0). Find the matrix for Dg(0, 1, 1), where g = (g1 , g2 ). 21. Consider the following Keynesian model in macroeconomics: Y = C[(1 − t)(Y + B/P )] + I(r, Y ) + G, M/P = L(Y, r), where Y (output) and r (interest rate) are endogenous and P (price), G (government expenditure), t (tax rate) , M (money supply), and B (government bonds) are exogenous. C, I, and L are functions for consumption, investment, and money demand respectively. Using Cramer’s rule, find the following partial derivatives in studying the comparative statics of the model. (a) ∂Y /∂G, (b) ∂r/∂M , (c) ∂Y /∂B. What assumptions do you have to make in order for the model to work? 22. Suppose a consumer wants to maximize the following function with respect to consumption ct and ct+1 : Vt = U (ct ) + βU (ct+1 ) 132

(5.12)

5.4. EXERCISES subject to the intertemporal budget constraint ct +

ct+1 yt+1 = yt + , 1+r 1+r

(5.13)

where U is the instantaneous utility function, β is a discount factor, yt and yt+1 are incomes in periods t and t + 1 respectively, and r is the interest rate. (a) By writing (5.12) as f (x, θ) = U (ct ) + βU (ct+1 ) − Vt with x = ct+1 and θ = ct , apply the implicit function theorem to find the marginal rate of time preference, dct+1 /dct . (b) Find the slope dct+1 /dct of the intertemporal budget constraint (5.13) as well. (c) Combine your results to get the Euler equation: βU 0 (ct+1 ) (1 + r) = 1. U 0 (ct )

(5.14)

The Euler equation (5.14) is the cornerstone of the dynamic general equilibrium model used in macroeconomic analysis. 23. (Wickens, 2011, chapter 8) In the transaction cost approach to money demand, households incur a transaction cost in consumption. The following two conditions are satisfied in the steady state: c + πm + T (c, m) − x − θb = 0,

(5.15)

Tm (c, m) + R = 0,

(5.16)

where c is consumption, m is money demand, x is exogenous income, b is bond holding, and R is the market interest rate. The transaction cost T is a function of c and m, which has the following properties: T (0, m) = 0, Tc ≥ 0, Tcc ≥ 0, Tm ≤ 0, Tmm > 0, Tmc = 0, where Tc means ∂T /∂c, and Tmc means ∂ 2 T /∂m∂c, etc. Equations (5.15) and (5.16) can be written as f (x, y) = 0, where f : R2 × R3 → R2 , x = (c, m)T is the 133

CHAPTER 5. VECTOR CALCULUS vector of endogenous variables, and y = (x, b, R)T are exogenous. (a) Find the impact of an increase in interest rate on money demand. (b) Find the effect of a decrease in exogenous income x on consumption.

134

Chapter 6 Convex Analysis Models in economic analysis often assume sets or functions are convex. These convexity properties make solutions of optimization problem analytically convenient. In this chapter we discuss various concepts of convexity in Euclidean spaces and their implications. The readers can consult Rockafellar (1970) for more details.

6.1

Basic Definitions

Suppose that x, y are vectors in an n-dimensional vector space V , where n ≥ 2. If x and y are linearly independent, then the set of all linear combinations of the two vectors, S = span({x, y}) = {αx + βy : α, β ∈ R} is a two-dimensional subspace of V . If we impose the restriction β = 1 − α, the set M = {αx + (1 − α)y : α ∈ R} is called an affine set and is no longer two dimensional or a subspace. To see this, we can express a vector z ∈ M as z = αx + (1 − α)y = y + α(x − y), for some α ∈ R. Geometrically, x − y is the vector going from the head of y towards the head of x. Therefore z is a vector on the straight line passing through the heads of x and y. Since this line in general does not include the zero vector, M is not a subspace of V . A vector in M is called an affine combination of x and y. 135

CHAPTER 6. CONVEX ANALYSIS Now we further restrict the value of α to be between 0 and 1. Then the set C = {αx + (1 − α)y : α ∈ [0, 1]} is called the convex combinations of x and y. It is clear that this is the set of vectors lying between the heads of x and y. The idea can be extended to more than two vectors. Let A = {x1 , x2 , . . . , xn } be a set of vectors in V , then the convex combinations of A is defined as the set ( n ) n X X C= αi xi : αi ≥ 0, i = 1, . . . , n, αi = 1 . (6.1) i=1

6.2

i=1

Convex Sets

In chapter 3 we define a convex set in Rn under the discussion of a metric space. In general, a set S in a vector space V is convex if for any x, y ∈ S, the convex combinations of x and y are in S. Otherwise the set is called non-convex (see figure 6.1). The convex hull of a set A ⊆ V is defined to be the set of all convex combinations of vectors in A and is denoted by conv(A). That is, conv(A) = {αx + (1 − α)y : x, y ∈ A, α ∈ [0, 1]}. The convex hull of A is the smallest convex set in V that contains A. In other words, conv(A) is the intersection of all convex sets that contain A. When A is a finite set of say n vectors, conv(A) is the set defined in equation (6.1). Example 6.1. Let B = (e1 , e2 , . . . , en ) be the standard basis of Rn . Then the convex hull of B is called a standard simplex in Rn . Example 6.2. In consumer theory, the preference relation of a consumer is often assumed to be convex. More precisely, the upper contour set, also called the noworse-than set, of a consumption bundle a, % (a), is a convex set. Theorem 6.1. Let A and B be subsets of a vector space V . Then the convex hull of their Minkowski sum is the Minkowski sum of their convex hulls. That is, conv(A + B) = conv(A) + conv(B). The proof is left as an exercise. By the the principle of mathematical induction, 136

6.2. CONVEX SETS

x•



x

•y

y

• Figure 6.1: Convex and Non-Convex Sets

the theorem applies to any finite number of set, that is, for any sets A1 , A2 , . . . , An , ! n n X X conv Ai = conv(Ai ). (6.2) i=1

i=1

An important result in convex analysis is the Shapley-Folkman theorem, which roughly says that the sum of a large number of sets in an Euclidean space, not P necessary convex, is approximately convex. That is, the sum of sets ni=1 Ai for large n converges to the sum in equation (6.2). See Starr (2008) for a formal statement of the theorem. Example 6.3. We define the aggregate production set of an economy in example 4.9 as l X Y = Y1 + Y2 + · · · + Yl = Yj . j=1

The Shapley-Folkman theorem implies that even though the production sets Yi of individual firms are not convex, the aggregate production set Y can be assumed to be convex, making the analysis more tractable. Next we turn to the idea of a cone. A set S in a vector space V is called a cone if for any x ∈ S and α ≥ 0, the vector αx is in S. By definition a cone must contain the zero vector. A convex cone, as the name implies, is a cone that is also 137

CHAPTER 6. CONVEX ANALYSIS convex. Formally, a set S ∈ V is a convex cone if for any x, y ∈ S and α, β ≥ 0, αx + βy ∈ S. Example 6.4. Suppose that f : R → R is given by f (x) = |ax|, for a ∈ R. Then the epigraph of f is a convex cone in R2 . Example 6.5. In consumer theory, the consumption set of a consumer, Rn+ is a convex cone. In production theory, if a firm’s technology exhibits constant returns to scale, then the production set is a convex cone in Rn .

6.3

Affine Sets

6.3.1

Definitions

An affine set M is a subset of a vector space V such that for every x, y ∈ M and α ∈ R, αx + (1 − α)y ∈ M. The empty set ∅ and V are trivial examples of affine sets, and so is the set {x} which contains any single vector x ∈ V . In general, affine sets are “linear” sets such as a straight line or a plane in an Euclidian space. Theorem 6.2. An affine set which contains the zero vector is a subspace. Proof. It is clear that if S ∈ V is a subspace then it is an affine set with the zero vector. Conversely, let M be an affine set which contains 0. For M to be a subspace, we need to show that it is closed under addition and scalar multiplication. Let x, y ∈ M . Then for any α ∈ R, αx = αx + (1 − α)0 ∈ M. Also,   1 1 1 (x + y) = x + 1 − y∈M 2 2 2 so that

 x+y =2

 1 (x + y) ∈ M 2 138

6.3. AFFINE SETS as required. Given an affine set M , the translation of M by a vector a is defined as M + a = {x + a : x ∈ M }. It is easy to verify that M + a is also an affine set. In fact we say an affine set L is parallel to an affine set M if L = M + a for some a ∈ V . Theorem 6.2 implies that every non-empty affine set M is parallel to a unique subspace S, given by S = {x − a : x, a ∈ M }.

6.3.2

Hyperplanes and Half-Spaces

Let V be an n-dimensional vector space. Recall that the kernel of a linear functional f on V is a subspace of dimension n−1, which we called a hyperspace in section 4.3.3. The hyperplane we defined in (4.8) is in fact a translation of this hyperspace. We can look at hyperplanes again in the context of an inner product space. Let S be a one-dimensional subspace of V . Every basis of S consists of a single non-zero vector, say, b. The orthogonal complement of S is a subspace of dimension n − 1: S ⊥ = {x ∈ V : bT x = 0}.

(6.3)

A hyperplane is an affine set parallel to S ⊥ , that is, for any vector a ∈ V , S ⊥ + a = {x + a : bT x = 0} = {y : bT (y − a) = 0}

(6.4)

= {y : bT y = c}

(6.5)

where c = bT a. For example, by taking b = (p1 , . . . , pn ) in Rn and c = π, the hyperplane in (6.5) becomes the isoprofit hyperplane in (4.9). Notice that in (6.4) the hyperplane is orthogonal to the vector b and passes through the vector a. For any non-zero vector b ∈ V and any c ∈ R, two closed half-spaces can be defined as H + = {x ∈ V : bT x ≥ c} and H − = {x ∈ V : bT x ≤ c}. 139

CHAPTER 6. CONVEX ANALYSIS All half-spaces are convex sets. If c = 0, then the half-space becomes a convex cone. Obviously, H + ∩ H − is a hyperplane. Half-spaces are useful in defining feasible sets in economics. For example, the budget set B of a consumer can be represented by the intersection of n + 1 half-spaces: T B = {x : pT x ≤ M } ∩ {x : eT 1 x ≥ 0} ∩ · · · ∩ {x : en x ≥ 0}

where p is the market price vector of the n goods and M is the income of the consumer. Notice that the intersections of any number of half-space is a convex set.

6.4 6.4.1

Convex Functions Convex and Concave Functions

Let f : S → R be a functional on a subset S of an inner product space V . Recall that the epigraph of the function is defined as epi f = {(x, y) ∈ S × R : f (x) ≤ y}. The hypograph of f , on the other hand, is defined as hypo f = {(x, y) ∈ S × R : f (x) ≥ y}. It is obvious that the intersection of epi f and hypo f is the graph of f . We call f a convex function if epi f is a convex subset of V × R. The definition implies that the domain S of f is a convex set in V . If hypo f is a convex subset of V × R, then f is called a concave function. A function is called an affine function if it is both convex and concave. It is straight forward to show that f is convex if and only if −f is concave. An important characterization of convex functions is as follows. Theorem 6.3. A function f : S → R is convex if and only if for every x, z ∈ S and 0 < α < 1, f ((1 − α)x + αz) ≤ (1 − α)f (x) + αf (z). (6.6) Proof. Suppose that f is a convex function on S. By definition (x, f (x)) and (z, f (z)) are in epi f . Since epi f is convex, for 0 < α < 1, ((1 − α)x + αz, (1 − α)f (x) + αf (z)) = (1 − α)(x, f (x)) + α(z, f (z)) ∈ epi f, 140

6.4. CONVEX FUNCTIONS which implies (6.6). The converse is straight forward and is left as an exercise. Theorem 6.3 can be generalized to any convex combination: Theorem 6.4 (Jensen’s Inequality). A function f : S → R is convex if and only if P for every x1 , . . . , xm ∈ S and α1 ≥ 0, . . . , αm ≥ 0 with m i=1 αi = 1, f (α1 x1 + · · · + αm xm ) ≤ α1 f (x1 ) + · · · + αm f (xm ). The inequalities in Theorems 6.3 and 6.4 is reversed if f is a concave function. The inequalities turn into equalities if f is an affine function. In fact, every affine function can be expressed in the form f (x) = aT x + b where a ∈ V and b ∈ R. Example 6.6. The followings are some examples of convex function of single variable, which can be characterized by f 00 (x) ≥ 0. 1. f (x) = eax , a ∈ R; 2. f (x) = xa , for x ≥ 0 and a ≥ 1; 3. f (x) = − log x, for x > 0. Example 6.7. Applying Theorem 6.4 to the negative log function above, we have − log(α1 x1 + · · · + αm xm ) ≤ −α1 log(x1 ) − · · · − αm log(xm ). Multiplying both sides by −1 taking the exponential, we have α1 x1 + · · · + αm xm ≥ xα1 1 · · · xαmm .

(6.7)

Setting αi = 1/m for i = 1, . . . , m, the inequality in (6.7) shows that the arithmetic mean of m positive numbers is bounded below by their geometric mean. New convex functions can be constructed using simple convex functions. The following results are useful for this purpose and for recognizing convex functions. 1. Suppose f is a convex function on a convex set S in a vector space V and g : R → R is an increasing and convex function. Then g ◦ f is also a convex function. For example, if f is convex, then h(x) = ef (x) is convex. 141

CHAPTER 6. CONVEX ANALYSIS

Figure 6.2: Constructing a Convex Function from a Convex Set 2. If f and g are convex functions on a convex set S, then αf + g is also convex for α > 0. Consequently, the set of all convex functions on S is a cone. 3. Let T be a convex set in Rn+1 , and let S be the orthogonal projection of T on Rn . For every x ∈ S, define f (x) = inf{y : (x, y) ∈ T }. Then f is a convex function on S. This result is like the converse of the definition of a convex function. Here T is contained in the epigraph of f . The orthogonal projection of T on Rn becomes the domain of f . Figure 6.2 illustrates an example for n = 2. The underside of T is the graph of the convex function f . 4. Suppose f : V → W is a linear transformation and g is a convex function on W . Then g ◦ f is a convex function on V . The following theorem lists the different characterizations of a concave function. 142

6.4. CONVEX FUNCTIONS Theorem 6.5. Suppose f is a C 2 functional on an open convex set S ∈ Rn . The following statements are equivalent: 1. Function f is concave. 2. Function −f is convex. 3. The hypograph of f , {(x, z) ∈ S × R : f (x) ≥ z} is convex. 4. For every x, y ∈ S and α ∈ [0, 1], f (αx + (1 − α)y) ≥ αf (x) + (1 − α)f (y). 5. For every x, y ∈ S, (∇f (x) − ∇f (y))T (x − y) ≤ 0. 6. For every x, x0 ∈ S, f (x) ≤ f (x0 ) + ∇f (x0 )T (x − x0 ). 7. For every x ∈ S, the Hessian ∇2 f (x) is negative semi-definite.

6.4.2

Quasi-Convex and Quasi-Concave Functions

Economists sometimes find convex and concave functions are too restrictive. A more general class of functions includes the quasi-convex and quasi-concave functions. Let f : S → R be a functional on a subset S of an inner product space V . Then f is called a quasi-convex function if, for 0 ≤ α ≤ 1 and x, z ∈ S, f ((1 − α)x + αz) ≤ max{f (x), f (z)}. Similarly, f is called a quasi-concave function if, for 0 ≤ α ≤ 1 and x, z ∈ S, f ((1 − α)x + αz) ≥ min{f (x), f (z)}. Quasi-convex and quasi-concave functions are less restrictive than convex and concave functions. Some properties of these functions are listed below. More can be found in Carter (2001, p. 336–342). 143

CHAPTER 6. CONVEX ANALYSIS 1. If f is concave (convex), then it is quasi-concave (convex). 2. Monotone functions are both quasi-convex and quasi-concave. The following theorem lists the characterizations of quasi-concave functions. Theorem 6.6. Suppose f is a C 2 functional on an open convex set S ∈ Rn . The following statements are equivalent: 1. Function f is quasi-concave. 2. Function −f is quasi-convex. 3. For all x, y ∈ S and 0 < α < 1, f (αx + (1 − α)y) ≥ min{f (x), f (y)}. 4. For all a ∈ S, the upper contour set %f (a) = {x ∈ S : f (x) ≥ f (a)} is a convex set. 5. For all x, y ∈ S, f (y) ≥ f (x) implies that ∇f (x)T (y − x) ≥ 0. 6. For all x ∈ S and uT u = 1, ∇f (x)T u = 0 implies that (a) uT ∇2 f (x)u < 0, or (b) uT ∇2 f (x)u = 0 and g(t) = f (x + tu) is a quasiconcave function of one variable of t such that x + tu ∈ S. Figure 6.3 illustrates the classification of curvatures for the simple case of single variable functions. Parts (a) and (b) are strictly concave functions but (b) is monotone (increasing in this case). Part (c) is an affine function and therefore both concave and convex. Parts (d) and (e) are strictly convex with (d) being monotone. All except (e), however, are quasi-concave functions. Similarly, all except (a) are quasi-convex functions. Part (f) is neither convex nor concave but since it is monotone it is both quasi-convex and quasi-concave.

6.4.3

Increasing Functions on Rn+

If we want to define an increasing function f : Rn+ → R+ , we first need an order in the domain. A partial order on Rn+ can be defined as follows. For any x, y ∈ Rn+ , x ≥ y if xi ≥ yi for i = 1, . . . , n. In economics, we frequently use the following induced relations on Rn+ as well: 144

6.5. HOMOGENEOUS FUNCTIONS

y

y

y

6

6

6

- x

- x

- x

(a)

(b)

(c)

y

y

y

6

6

6

- x

- x

(d)

(e)

- x

(f)

Figure 6.3: Convex and Concave Functions 1. x > y if x ≥ y and x 6= y, 2. x  y if xi > yi for i = 1, . . . , n. For example, in consumer analysis, the non-satiation axiom implies that the utility function U of a consumer is increasing. That is, if x and y are two consumption bundles and x ≥ y, then U (x) ≥ U (y).

6.5

Homogeneous Functions

Let S be a cone in a vector space V . A functional f on S is said to be homogeneous of degree k if, for every α > 0 and x ∈ S, f (αx) = αk f (x). If k = 1, then f is called a linearly homogeneous function. For example, a CobbDouglas function of two variables with a general form f (x, y) = axβ y γ 145

CHAPTER 6. CONVEX ANALYSIS is homogeneous of degree β + γ. A functional f on a convex cone S is called homothetic if, for every α > 0 and x, z ∈ S, f (x) = f (z) implies that f (αx) = f (αz). The order % on S induced by a homothetic function is often called a homothetic preference relation. It follows that if x ∼ z then αx ∼ αz. In economics we frequently encounter linearly homogeneous functions. For example, in production theory, a cost function C(p, y), where p is the price vector of the input factors and y is the quantity of output, is linearly homogeneous in p.1 That is, C(αp, y) = αC(p, y). All linear functions are linearly homogeneous but the converse is not true. For Q P example, the Cobb-Douglas function, f (x) = i xβi i , βi > 0, i βi = 1, is linearly homogeneous but not linear. Nevertheless, linearly homogeneous functions have properties that closely resemble linear functions. The following result illustrates this point. Theorem 6.7 (Euler’s Theorem). Suppose that f : Rn+ → R is a linearly homogeneous C 2 functional, that is, f (αx) = αf (x) (6.8) for α > 0. Then for all x ∈ Rn++ , 1. f (x) = ∇f (x)T x, 2. ∇2 f (x)x = 0. Proof. Differentiate both sides of (6.8) with respect to α we have [∇f (αx)]T x = f (x). Setting α = 1 gives the first result. Now differentiate both sides of (6.8) with respect to x gives α∇f (αx) = α∇f (x) 1 In the context of consumer theory y is replaced by a given utility level u and C is often called an expenditure function.

146

6.6. SEPARATING HYPERPLANES or ∇f (αx) = ∇f (x). Now differentiate the above equation with respect to α to get ∇2 f (αx)x = 0. Setting α = 1 gives the second result. Apply the first part of Euler’s theorem to the cost function, we have C(p, y) = [∇p C(p, y)]T p.

(6.9)

n Since by definition C(p, y) = h(p, y)T p, where h : Rn+1 ++ → R+ is the Hicksian demand function for the inputs, and (6.9) holds for any p ∈ Rn++ , we have ∇p C(p, y) = h(p, y), which is often called Shephard’s Lemma. Apply the second part of Euler’s theorem we have ∇2p C(p, y)p = 0,

or Dp h(p, y)p = 0. This can be written as

n X ∂hi (p, y) j=1

∂pj

pj = 0

(6.10)

for i = 1, . . . , n. Since each Hicksian demand function is decreasing in its own price, ie, ∂hi (p, y)/∂pi < 0, equation (6.10) implies that at least one of ∂hi (p, y)/∂pj must be positive. In other words, at least one of the other n − 1 inputs is a substitute for input i. In the case of n = 2, the two inputs normally are substitutes of each other.2

6.6

Separating Hyperplanes

Suppose A and B are convex subsets of V . Hf (c) is said to be a separating hyperplane of A and B if A is contained in one of the half-space defined by Hf (c) and 2 An exception is the Leontief production function, f (x) = min{x1 /a1 , . . . , xn /an }, where a1 , . . . , an > 0. In this case ∇2p C(p, y) is a zero matrix. Note that f is not differentiable in this case.

147

CHAPTER 6. CONVEX ANALYSIS B is contained in the other half-space. In other words, for every a ∈ A and b ∈ B, f (a) ≤ c ≤ f (b) or f (b) ≤ c ≤ f (a). If S is a convex subset of V , then Hf (c) is called a supporting hyperplane of S if S is contained in one of the half-spaces of Hf (c) and at least one vector of S lies in the hyperplane itself. That is, for every x ∈ S, f (x) ≥ c or f (x) ≤ c and there exists s ∈ Hf (c) ∩ S. For example, in production theory, the cost minimization problem involves finding the supporting hyperplane of the upper contour set of the production function that is orthogonal to the input price vector p. The following result is a form of separating hyperplane theorem. Theorem 6.8. Suppose S is a closed convex subset of a normed vector space V and z∈ / S. There exists a vector p ∈ V and a vector s ∈ S such that for every x ∈ S, pT x ≥ pT s > pT z.

(6.11)

In Theorem 6.8, s is the point in S that is “closest” to z and p is the vector s − z. Then z and S are separated by a hyperplane orthogonal to p and passing through any point between z and s. The hyperplane passing through s, using the form expressed in (6.4), is Hf (c) = {y : pT (y − s) = 0} where c = pT s. In fact, (6.11) means that Hf (c) is a supporting hyperplane of S, with the point z contained in the other half-space. It is important to emphasize that S must be convex, otherwise one of the half-space defined by Hf (c) may not contain S. Other relating results are listed below, in the context of a normed vector space V: 1. Minkowski’s Theorem: A closed convex set is the intersection of its supporting half-spaces. 2. Two disjoint closed convex sets have a separating hyperplane. 148

6.7. EXERCISES 3. Two closed convex sets which intersect at a single point only have a common separating hyperplane which is also a supporting hyperplane of both sets.

6.7

Exercises

1. Show that a set A in a vector space is convex if and only if A = conv(A). 2. Let S and T be convex subsets of a vector space V . (a) Prove that S + T is convex. (b) Prove that S ∩ T is convex. (c) Is S ∪ T convex? 3. Show that an open ball in Rn is convex. 4. Let f : V → W be a linear transformation. Suppose that S ⊆ V and T ⊆ W are convex sets. Show that f (S) and f −1 (T ) are convex. 5. Show that the orthogonal projection of a convex set on a subspace is also convex. 6. Let F (S, R) be the set of all functionals on a metric space S. Let C(S, R) be the subset of continuous functions in F (S, R). Prove or disprove: C(S, R) is a convex set in F (S, R). 7. Let L(V ) be the set of all linear operators on a finite-dimensional vector space V . Prove or disprove: The set of all invertible operators in L(V ) is a convex set. 8. Let A be a subset of a vector space V . Suppose that 0 ≤ α ≤ 1. Prove that A is convex if and only if A = αA + (1 − α)A. 9. Prove theorem 6.1 on the convex hull of Minkowski sum. 10. When is a cone S in a vector space V a subspace of V ? Is S a convex cone? 11. Prove the statement in example 6.4. 12. Suppose S and T are convex cones in a vector space V . Show that S + T is also a convex cone. 13. Let F (X) be the set of all functionals on a set X. Show that the set of all increasing functionals on X is a cone in F (X). 149

CHAPTER 6. CONVEX ANALYSIS 14. Let S be a subset of a vector space V . (a) Prove or disprove: If S is a convex cone, then it is a subspace of V . (b) Is the converse true? Explain. 15. The conic hull of a set A in a vector space V is the smallest cone in V that contains A, and is denoted by cone(A). (a) Give a formal definition of the conic hull. (b) Let V = R2 , x = (−1, 0), y = (−1, −1), and let A = conv({x, y}). What is cone(A)? (c) Sketch the region of cone(A) ∈ R2 in a diagram. 16. Show that S ⊥ defined in (6.3) is a subspace of V . 17. Let a and b be two distinct vectors in a vector space V . Show that A = {x : aT x ≤ 0} ∩ {x : bT x ≤ 0} is a convex cone. 18. List the counter parts of theorem 6.5 for a convex function. 19. Let f : X → R be a convex function on a metric space X and g : R → R be a convex and increasing function. Show that g ◦ f is convex. 20. Let f : Rn+ → R+ be an increasing and concave function. Show that if there exists a z  0 such that f (z) = 0, then f (x) = 0 for all x ∈ Rn+ . (Hint: First show that for all α ≥ 0, f (αz) = 0. Then for any x ∈ Rn+ , we can choose a large enough α such that αz  x.) 21. Let f be a linearly homogeneous function on a convex cone S. Show that if f is convex, then for all x, z ∈ S, f (x + z) ≤ f (x) + f (z). 22. Let f : Rn+ → R be a C 1 homogeneous function of degree k. Show that kf (x) = ∇f (x)T x. 150

6.7. EXERCISES 23. Let S be a convex cone in Rn+ . Suppose that h : S → R is a linearly homogeneous function and g : R → R is a strictly increasing function. Show that f = g ◦ h is homothetic. 24. Prove Minkowski’s Theorem. 25. Find a separating hyperplane of S = {(x1 , x2 ) ∈ R2 : x1 x2 ≥ 1, x1 ≤ 0, x2 ≤ 0} and the origin. Find the supporting hyperplane of S that is orthogonal to (1, 1). 26. Suppose that the function f : R2 → R is given by f (x, y) = y − x2 . (a) Let S = %f (0) = {(x, y) : f (x, y) ≥ 0} be the upper contour set of the value 0 induced by f (see section 2.5). Is S a convex set? Explain. (b) Find two supporting hyperplanes of S which pass through the point z = (−3, 8). 27. Let L(V ) be the set of linear operators on an n-dimensional vector space V . Show that the determinant function D : L(V ) → R is homogenous of degree n. 28. Suppose that f is a linear transformation from a vector space V to a vector space W . Show that the graph of f is a subspace in the product set V × W .

151

CHAPTER 6. CONVEX ANALYSIS

152

Chapter 7 Optimization 7.1

Unconstrained Optimization

In economics we frequently have to find the points that maximize or minimize a differentiable functional f (x, θ). Here f is called the objective function, x ∈ Rn is the control or endogenous variable, and θ ∈ Rl is the parameter or exogenous variable. The choice of x may be constrained to a subset G(x, θ) of Rn , which is often called the feasible set. In a maximization process, we write the problem as max f (x, θ) x

subject to x ∈ G(x, θ). Let us ignore the parameter θ for now. Suppose that x∗ is a local maximum of f . That is, there exists a neighbourhood B containing x∗ such that for every x ∈ G∩B, f (x∗ ) ≥ f (x).

(7.1)

The linear approximation of f (x) about x∗ is f (x) = f (x∗ ) + ∇f (x∗ )T (x − x∗ ). Putting (7.2) into (7.1) gives f (x∗ ) ≥ f (x∗ ) + ∇f (x∗ )T (x − x∗ ), 153

(7.2)

CHAPTER 7. OPTIMIZATION or ∇f (x∗ )T (x − x∗ ) ≤ 0. Notice that the expression on the left hand side of the above inequality is the directional derivative of f at x∗ in the direction of (x − x∗ ) multiplied by its length. If x∗ is an interior point of G, then the inequality becomes an equality. Otherwise if there exists an x such that ∇f (x∗ )T (x − x∗ ) < 0, then the directional derivative in the opposite direction is positive, contradicting the fact that x∗ is a local maximum. Therefore the necessary condition for an unconstrained maximization is ∇f (x∗ ) = 0. (7.3) Any point that satisfies (7.3) is called a stationary point of f . In the special case that the control variable is constrained to Rn+ , the necessary condition can be summarized by the following complementary slackness condition: ∇f (x∗ ) ≤ 0,

x∗ ≥ 0,

∇f (x∗ )T x∗ = 0.

Example 7.1. Suppose f (x, y) = x2 − y 2 . Then ∇f (x, y) = (2x − 2y)T so that ∇f (0) = 0. It is, however, obvious in Figure 7.1 that the origin is neither a maximum nor a minimum (it is called a saddle point).

The above example shows that we need to find a sufficient condition for a local maximum. It turns out that such a condition exists for an interior strict local maximum. That is, f (x∗ ) > f (x) for all x in the neighbourhood of x∗ . The quadratic (Taylor) approximation of f (x) about x∗ is 1 f (x) = f (x∗ ) + ∇f (x∗ )T (x − x∗ ) + (x − x∗ )T ∇2 f (x∗ )(x − x∗ ) 2 1 ∗ ∗ T 2 ∗ = f (x ) + (x − x ) ∇ f (x )(x − x∗ ) 2 where in the first equality above ∇f (x∗ )T (x − x∗ ) = 0 from the necessary condition in (7.3). Therefore f (x∗ ) > f (x) if and only if (x − x∗ )T ∇2 f (x∗ )(x − x∗ ) < 0. 154

7.2. EQUALITY-CONSTRAINED OPTIMIZATION

Figure 7.1: Graph of the saddle z = x2 − y 2 This means that the Hessian ∇2 f (x∗ ) must be negative definite. The above analysis deals with local maximum. If the objective function f is a concave function, then (7.3) becomes a necessary and sufficient condition for a global maximum.

7.2

Equality-Constrained Optimization

We want to solve the following problem: max f (x) x

subject to g(x) = 0. Here f : Rn → R is the objective function and g : Rn → Rm is the constraint function. Note that m < n otherwise it is over constrained and Dg(x) cannot have full rank m. The following result characterizes the necessary condition for a 155

CHAPTER 7. OPTIMIZATION maximum. Theorem 7.1 (Lagrange Multiplier Theorem). Let f : S ⊆ Rn → R where S is open and g : S → Rm be C 2 functions. Suppose x∗ is a local maximum of f on the set G = S ∩ {x : g(x) = 0}. Suppose also that Dg(x∗ ) has rank m. Then there exists a unique vector λ ∈ Rm such that ∇f (x∗ ) = Dg(x∗ )T λ. In practice, we define a Lagrangian function L : S × Rm → R such that L(x, λ) = f (x) − g(x)T λ. Then we set the gradients of L with respect to x and λ to zero as if we are maximizing L, that is, ∇x L(x∗ , λ) = ∇f (x∗ ) − Dg(x∗ )T λ = 0 (7.4) and ∇λ L(x∗ , λ) = −g(x) = 0, which give us the first order conditions as in the Lagrange Multiplier Theorem. The second order conditions are a bit more complicated than those for the unconstrained optimization. First recall that the kernel of the linear function Dg(x∗ ) is defined as K = {h ∈ Rn : Dg(x∗ )h = 0}. Geometrically, the kernel K is the linear approximation of the constraint g(x) = 0 at the point x∗ . The following theorem states that the Hessian of L has something to do with the local curvature of f at x∗ when x is constrained on this linear approximation of g. Since g is a vector-valued function, we have to express it in terms of its components in order to write down the second derivative. Hence we write g as g(x) = (g1 (x) g2 (x) · · · gm (x))T . Then (7.4) can be written as ∗



∇x L(x , λ) = ∇f (x ) −

m X

λi ∇gi (x)

i=1

so that we can let ∇2 L denote the n × n symmetric matrix ∇2x L(x∗ , λ)

2



= ∇ f (x ) −

m X i=1

156

λi ∇2 gi (x).

7.2. EQUALITY-CONSTRAINED OPTIMIZATION Theorem 7.2. Suppose there exist points x∗ ∈ G and λ ∈ Rm such that Dg(x∗ ) has rank m and ∇f (x∗ ) = Dg(x∗ )T λ. 1. If f has a local maximum on G at x∗ , then hT ∇2 Lh ≤ 0 for all h ∈ K. 2. If hT ∇2 Lh < 0 for all h ∈ K with h 6= 0, then x∗ is a strict local maximum of f on G. Note that in Theorem 7.2 part (a) is a necessary condition and part (b) is a sufficient condition. Also the definiteness of ∇2 L can only be confirmed on the kernel K of Dg(x∗ ). The question now is to find a way to characterize the definiteness of ∇2 L on K. For this we need the bordered Hessian at x∗ , which is defined as Bj for j = 1, . . . , n:   0 ··· 0 ∂g1 (x∗ )/∂x1 ··· ∂g1 (x∗ )/∂xj   .. .. .. .. ... ...   . . . .    ∗ ∗ 0 ··· 0 ∂gm (x )/∂x1 ··· ∂gm (x )/∂xj    Bj =  . ∗ ∗ 2 ∗ 2 2 ∗ ∂g1 (x )/∂x1 · · · ∂gm (x )/∂x1 ∂ L(x , λ)/∂x1 · · · ∂ L(x , λ)/∂x1 ∂xj    .. .. .. .. .. ..   . . . . . .   ∗ ∗ 2 ∗ 2 ∗ 2 ∂g1 (x )/∂xj · · · ∂gm (x )/∂xj ∂ L(x , λ)/∂xj ∂x1 · · · ∂ L(x , λ)/∂xj For notational convenience, we let Dg(x∗ )j be the m × j matrix which includes the first j columns of Dg(x∗ ), and ∇2 Lj be the j × j matrix which includes the first j rows and columns of ∇2 L, then ! 0m×m Dg(x∗ )j Bj = . Dg(x∗ )T ∇ 2 Lj j Therefore Bj is a (m + j) × (m + j) symmetric square matrix, and Bn is the full bordered Hessian with the whole Dg(x∗ ) and ∇2 L. Next, for the set of integers N = {1, 2, . . . , n}, let π be a permutation of N . Then there are exactly n! of such permutations. If we label the rows and columns of ∇2 L by the numbers in N we can obtain n! different matrices by applying a permutation π to the rows and columns at the same time. Similarly, π can be applied to the columns of Dg(x∗ ). Hence we denote ∇2 Lπ and Dg(x∗ )π the permutation matrices for the permutation π, and the permuted bordered Hessian ! ∗ π 0 Dg(x ) m×m j Bjπ = (Dg(x∗ )πj )T ∇2 Lπj 157

CHAPTER 7. OPTIMIZATION for j = 1, . . . , n. Now we can states the second-order conditions for equality constrained maximization: Theorem 7.3. Let Bj be the bordered Hessian defined above. Then 1. hT ∇2 Lh ≤ 0 for all h ∈ K if and only if for all permutation π of N and for j = 1, . . . , n, we have (−1)j |Bjπ | ≥ 0, 2. hT ∇2 Lh < 0 for all h ∈ K with h 6= 0 if and only if for j = 1, . . . , n, we have (−1)j |Bj | > 0. The proof of this theorem can be found in Debreu (1952).

Example 7.2. Find the maximum of the function f (x, y) = x2 −y 2 in the constraint set {(x, y) ∈ R2 : x2 + y 2 = 1}. Here n = 2, m = 1, with g(x, y) = x2 + y 2 − 1. The Lagrangian is L(x, y, λ) = x2 − y 2 − λ(x2 + y 2 − 1). The first-order condition is  2x − 2λx   ∇L(x, y, λ) =  −2y − 2λy  = 0. 1 − x2 − y 2 

The solutions to the equation are (x∗ , y ∗ , λ) = (1, 0, 1), (−1, 0, 1), (0, 1, −1), and (0, −1, −1). Evaluating f at these four points gives f (1, 0) = f (−1, 0) = 1 and f (0, 1) = f (0, −1) = −1. Therefore (1, 0) and (−1, 0) are the maximum points. We can verify this by the second-order condition. Now ! 2 − 2λ 0 ∇2x L(x∗ , λ) = 0 −2 − 2λ and Dg(x∗ , y ∗ ) = (2x∗ , 2y ∗ ). The bordered Hessian matrices at the above four points are, respectively,         0 2 0 0 −2 0 0 0 2 0 0 −2         0  , 0 4 0 ,  0 4 0  . 2 0 0  , −2 0 0 0 −4 0 0 −4 2 0 0 −2 0 0 Table 7.1 lists the values of (−1)j |Bj |, j = 1, 2 for the four points, which confirms our results. 158

7.2. EQUALITY-CONSTRAINED OPTIMIZATION

Table 7.1: Values of (x∗ , y ∗ ) −|B1 | (1, 0) 4 (−1, 0) 4 (0, 1) 0 (0, −1) 0

(−1)j |Bj | |B2 | 16 16 −16 −16

The function in the above example is called a saddle (see Figure 7.1). The example in section 7.1 shows that the function has no maximum nor minimum. Maxima and minima do exist, however, on the constrained set of the unit circle as demonstrated.

Example 7.3. In economics we frequently model a consumer’s utility function with an increasing and quasi-concave C 2 function u = f (x). If we assume an interior solution, the utility maximization problem becomes a equality constraint problem. The constraint of course is the budget hyperplane g(x) = pT x − y, where p ∈ Rn++ is the price vector and y > 0 income. It follows from Table 5.1 that ∇g(x) = p. Recall from Figure 5.1 that the gradient of a functional is orthogonal to the tangent hyperplane of the level surface at any point. Therefore p is orthogonal to the budget hyperplane itself. Also, ∇f (x) is the vector orthogonal to tangent hyperplane of the level surface of f (in economics we call it an indifference surface) at any consumption bundle x. The Lagrange Multiplier Theorem implies that at the optimal bundle x∗ , ∇f (x∗ ) = λp, that is, ∇f (x∗ ) and p are pointing in the same direction. Another way to say this is at x∗ , the budget hyperplane is tangent to the indifference surface, or, in the case of two goods, the marginal rate of substitution (MRS) is equal to the price ratio p1 /p2 . 2 Next we put the the parameters θ back into the objective function and the constraints. The maximum points is then a function of of the parameters, x∗ = φ(θ), so that f (x∗ ) = f (φ(θ)) which we call the value function. We want to study the effect on the value function when we change θ. In economics this is called comparative statics. The following result is very useful. 159

CHAPTER 7. OPTIMIZATION Theorem 7.4 (Envelope Theorem). Given the maximization problem max f (x, θ) x

subject to

g(x, θ) = 0

where θ ∈ Rl is a vector of parameters. Suppose x∗ ∈ G = {x : g(x, θ) = 0} is a local maximum. Then ∂f (x∗ , θ) ∂L(x∗ , θ) = , ∂θi ∂θi

i = 1, . . . , l

or in matrix form, ∇θ f (x∗ , θ) = ∇θ L(x∗ , θ) where L(x∗ , θ) = f (x∗ , θ) − g(x∗ , θ)T λ is the Lagrangian, and λ ∈ Rm is the vector of Lagrange multipliers.

Proof. Since x∗ depends on the parameter θ we can write x∗ = φ(θ) where φ : Rl → Rn . Since g(x∗ , θ) = 0, λT g(x∗ , θ) = 0. Subtracting this term from f (x∗ , θ) does not change its value, that is, f (φ(θ), θ) = f (φ(θ), θ) − λT g(φ(θ), θ). Differentiating both sides the above equation with respect to θ and using the chain rule, we have ∇θ f (φ(θ), θ) = Dφ(θ)T ∇x f (x∗ , θ) + ∇θ f (x∗ , θ) − [Dx g(x∗ , θ)Dφ(θ) + Dθ g(x∗ , θ)]T λ   = Dφ(θ)T ∇x f (x∗ , θ) − Dx g(x∗ , θ)T λ + ∇θ f (x∗ , θ) − Dθ g(x∗ , θ)T λ = ∇θ f (x∗ , θ) − Dθ g(x∗ , θ)T λ = ∇θ L(x∗ , θ) as required. In the second equality above ∇x f (x∗ , θ) − Dx g(x∗ , θ)T λ = 0 by the Lagrange multiplier theorem. 160

7.3. INEQUALITY-CONSTRAINED OPTIMIZATION

7.3

Inequality-Constrained Optimization

In many economics applications the equality constraints are too restrictive. For example, in consumer analysis, instead of restricting a consumer to select bundles on the budget hyperplane, it may be desirable to expand the feasible set to a set in Rn defined by the intersection of the half-spaces pT x ≤ M, x1 ≥ 0, . . . , xn ≥ 0. The intersection of these n+1 half-spaces is a compact and convex set called a polytope. Formally, an inequality-constrained optimization problem can be stated as max f (x) x

subject to g(x) ≤ 0, where f : Rn → R and g : Rn → Rm are C 1 functions. Notice that there is no limitation on m as long as the feasible set G = {x : g(x) ≤ 0} is not empty. Suppose x∗ ∈ G is a local maximum of f and M = {1, . . . , m}. We can classify the components of g, which can be written as g1 (x) ≤ 0, . . . , gm (x) ≤ 0 into two groups. The first group B(x∗ ) ⊆ M , called the set of binding constraints, is defined by those components that the constraints at x∗ are equalities, that is, B(x∗ ) = {i : gi (x∗ ) = 0}. The second group, called the set of slack constraints, is defined by those components of g at x∗ with strict inequalities, S(x∗ ) = {j : gj (x∗ ) < 0}.

For all j ∈ S(x∗ ), the constraint gj (x∗ ) < 0 is non-binding. It means that there exists a neighbourhood of x∗ contained by the intersection of these constraints, which has no effect on the maximum status of x∗ even if the constraints are relaxed. On the other hand, for all i ∈ B(x∗ ), x∗ is the solution to the problem max f (x) x

subject to gi (x) = 0, 161

i ∈ B(x∗ ).

(7.5)

CHAPTER 7. OPTIMIZATION By the Lagrange Multiplier Theorem, there exists λi ’s such that ∇f (x∗ ) =

X

λi ∇gi (x∗ ).

(7.6)

i∈B(x∗ )

The signs of the λi ’s can be asserted by considering the following maximization problem: max f (x) x

i ∈ B(x∗ ).

subject to gi (x) = θi ,

(7.7)

Each of the binding equation in (7.5) is relaxed by replacing the zero by the parameter θi in (7.7). Using the envelope theorem, for each i ∈ B(x∗ ), ∂f (φ(θ), θ) = λi . ∂θi Since relaxing the constraints will only increase the value of f (φ(θ), θ), λi ≥ 0 for all i ∈ B(x∗ ). Since gi (x∗ ) = 0, we have λi gi (x∗ ) = 0,

i ∈ B(x∗ ).

Now for all j ∈ S(x∗ ), we can set λj = 0 so that λj gj (x∗ ) = 0,

j ∈ S(x∗ )

as well. With λj = 0 for all j ∈ S(x∗ ), (7.6) can be written as X

∇f (x∗ ) =

=

λi ∇gi (x∗ ) +

i∈B(x∗ ) m X

X

λj ∇gj (x∗ )

j∈S(x∗ )

λi ∇gi (x∗ )

i=1

= Dg(x∗ )T λ, where λ ∈ Rm . The signs of the λi ’s and gi (x∗ )’s can be summarized by the complementary slackness condition: For i = 1, . . . , m, λi ≥ 0,

gi (x∗ ) ≤ 0, 162

λi gi (x∗ ) = 0.

7.3. INEQUALITY-CONSTRAINED OPTIMIZATION We have shown the following result. Theorem 7.5 (Kuhn-Tucker Theorem). Let f : Rn → R and g : Rn → Rm be C 1 functions. Suppose that x∗ is a local maximum of f on the set G = {g(x) ≤ 0}. Then there exists a unique vector λ ∈ Rm such that ∇f (x∗ ) = Dg(x∗ )T λ; λ ≥ 0;

and

g(x∗ )T λ = 0.

In practice we construct the Lagrangian function L(x, λ) = f (x) −

m X

λi gi (x)

i=1

as before but we set the gradient of L with respect to x only to zero and not λ. Instead we impose the complementary slackness conditions. Therefore the necessary conditions for inequality constrained maximization are

λ ≥ 0;

∇x L(x∗ , λ) = 0;

(7.8)

g(x∗ ) ≤ 0;

(7.9)

g(x∗ )T λ = 0.

Example 7.4. Solve the optimization problem max x,y

log x + log(y + 5)

subject to x + y ≤ 4, x ≥ 0, y ≥ 0. We have n = 2, m = 3, and g(x, y) = (x + y − 4, −x, −y). The Lagrangian function is L(x, y, λ1 , λ2 , λ3 ) = log x + log(y + 5) − λ1 (x + y − 4) + λ2 x + λ3 y. 163

CHAPTER 7. OPTIMIZATION The necessary conditions in (7.8) and (7.9) are ∂L/∂x = 1/x − λ1 + λ2 = 0,

(7.10)

∂L/∂y = 1/(y + 5) − λ1 + λ3 = 0,

(7.11)

λ1 ≥ 0, x + y ≤ 4,

(7.12)

λ1 (x + y − 4) = 0,

λ2 ≥ 0, x ≥ 0,

λ2 x = 0

(7.13)

λ3 ≥ 0, y ≥ 0,

λ3 y = 0

(7.14)

We solve the above system by trials and errors: 1. Assume that we have an interior solution, that is, x > 0 and y > 0. Then by (7.13) and (7.14) λ2 = λ3 = 0. 2. Assume that in (7.12) x + y = 4 and λ1 ≥ 0. Then (7.10) and (7.11) imply that 1/x = λ1 , and 1/(y + 5) = λ1 , which gives x = y + 5. With the assumption that x + y = 4, we have y = −1/2, which contradicts the assumption that y > 0. 3. The above result gives us a hint that we should assume x > 0 and y = 0. Then λ2 = 0 and λ3 ≥ 0. We also keep the assumption that x + y = 4 and λ1 ≥ 0. With (7.10) and (7.11) we have 1/x − λ1 = 0, 1/5 − λ1 + λ3 = 0, x = 4. These give λ1 = 1/4 and λ3 = 1/20. Putting the results together, the solution (x∗ , y ∗ , λ1 , λ2 , λ3 ) = (4, 0, 1/4, 0, 1/20) satisfies the implications of the Kuhn-Tucker Theorem, with f (x∗ , y ∗ ) = log 4 + log 5. As an exercise you should try the assumptions that x = 0 and y > 0. 164

7.4. EXERCISE

7.4

Exercise

1. Let f : S → R where S ⊆ Rn is a convex set. Suppose f is a C 1 concave function and ∇f (x∗ ) = 0. Show that x∗ ∈ S is a global maximum for f over S. 2. Consider the functional f (x, y) = xy + 3y − x2 − y 2 . (a) Find the stationary point(s) of f . (b) Is it a maximum, minimum, or neither? Explain. 3. Find the stationary point(s) of f (x, y) = 1 − (x − 1)2 − (y − 1)2 . Determine whether the point(s) is (are) maximum, minimum, or neither. 4. Solve the following optimization problems. Check second order conditions. (a) maxx1 ,x2 −2x21 + x1 x2 − 2x22 − 3x1 − 3x2 , (b) minx1 ,x2 ,x3 x21 + x22 + x23 . 5. Find and classify the critical points of the function f (x, y) = x sin y. 6. A swimming pool is to be constructed with reinforced concrete. The pool will have a square base with length and width x and height h. The cost of the concrete, including labour and materials, forming the base is p per square metre. The concrete for the side walls costs q per square metre. The budget for the construction cost of the pool is M . (a) Setting up a constrained-optimization problem to maximize the volume of the pool. (b) Find the Lagrangian function. (c) Find the effect on the optimized volume with respect to a change in the cost of the base material p. 165

CHAPTER 7. OPTIMIZATION 7. The owner of a toy store can purchase yo-yos from the manufacturer for $4.20 each. The owner estimates that 80 will be sold at a price of $8.00 and that 10 more will be sold for every $0.50 decrease in price. What price should be charged to maximize profit? 8. Consider the unconstrained maximization of the objective function f (x, θ) where x ∈ Rn is the control variable and θ ∈ Rl is the parameter vector. The solution can be expressed as x∗ = φ(θ). Find the impact of a change in θ on x∗ , that is, find Dφ(θ). (Hint: Apply the implicit function theorem to the first order condition.) 9. Provide an alternative proof of the Cauchy-Schwarz Inequality: Let x, y ∈ Rn . Then (xT y)2 ≤ (xT x)(yT y) (7.15) Hint: If either x or y = 0, then (7.15) is trivially true. Assume x and y 6= 0, and therefore (xT x) > 0 and (yT y) > 0. Now for every real number α, define z = x + αy so that z ∈ Rn . We have 0 ≤ zT z = (x + αy)T (x + αy) = α2 yT y + 2αxT y + xT x = f (α).

(7.16)

The inequality in (7.16) is true for every real number α. Now minimize f (α) with respect to α. Denote the minimum point as α∗ . Calculate f (α∗ ) and it will turn out that the inequality f (α∗ ) ≥ 0 is equivalent to (7.15). 10. Suppose y ∈ R has a linear relation with x ∈ Rk , that is, y = xT β, where β ∈ Rk is the unknown matrix representation of a linear functional. To estimate the value of β we carry out n(> k) random experiments with different values of x and observe the corresponding values of y. The result can be presented in the following matrix form: y = Xβ +  where y ∈ Rn , X is a n × k matrix, and  ∈ Rn is the vector of random variables 166

7.4. EXERCISE caused by measurement errors. The error term of each observation i is assumed to have the same distribution with zero mean and variance σ 2 . Also, assume that E(i j ) = 0 for i 6= j. One way to estimate β is to minimize the sum of squared errors, that is, we want to minimize T . (a) Derive an expression for T . (b) Find the stationary point of T  with respect to β. Assuming X have rank k, derive the so-called least square estimator for β. (c) Find the Hessian of T . Show that it is positive definite so that you have indeed found a minimum point. 11. Suppose that 1 1 f (x) = x4 + ax2 + bx, 4 2 where a and b are parameters. (a) Find the set of critical points of f . (b) Apply the implicit function theorem to the necessary condition in part (a) to find the rate of change of the critical point x with respect to a and b. (c) Find the set of points (a, b) such that the implicit function theorem fails to apply. The set in part (a) is called a cusp manifold (Figure 7.2). The set in part (c) is called the bifurcation set. The function f is useful in the study of stability of dynamical systems. See, for example, Poston and Stewart (1978, p. 78–83) for details. 12. Verify that the other two points in Table 7.1 are minimum points. 13. In the constrained optimization problem max f (x, θ), x∈G(θ)

suppose that f is strictly quasi-concave in x and G(θ) is convex. Prove that a local maximum is a strict global maximum. 14. Given the production function f (x) = (x1 x2 )1/2 where x1 , x2 are input quantities, a firm is trying to minimize the input cost given a particular output level y. Suppose the market prices for x1 and x2 are $2 and $5 per unit respectively. 167

CHAPTER 7. OPTIMIZATION

Figure 7.2: A Cusp Manifold

168

7.4. EXERCISE (a) Set up the optimization problem. (b) Find the response of the cost with respect to an increase in output. 15. Find the bordered Hessian of the consumer maximization problem in example 7.3. 16. Let f : Rn+ → R+ be the C 2 production function of a firm. For y ∈ f (Rn+ ) and p ∈ Rn++ , define the producer’s cost function by C(p, y) = min{pT x : f (x) = y}. x

Prove that C satisfies the following three properties: (a) C(αp, y) = αC(p, y) for α > 0; (b) p1 ≤ p2 implies C(p1 , y) ≤ C(p2 , y); and (c) C(p, y) is concave in p for fixed y. 17. Consider the cost minimization problem min z

w1 z1 + w2 z2

subject to y = z1 z2 . where z ∈ R2+ and w ∈ R2++ . Show that the solution is f (z1∗ , z2∗ ) = 2(w1 w2 y)1/2 by representing it as an unconstrained minimization problem in one variable. 18. Find the maxima and minima of the function f (x, y) = 1/x + 1/y subject to the constraint 1/x2 + 1/y 2 = 1/4. 19. Solve the following constrained maximization problem: max z

1 a0 − aT z + zT Az 2

subject to DT z = b where D is a n × m matrix of rank m, A = AT is a negative definite n × n matrix, and m < n. Check whether your solution satisfies a set of second order sufficient conditions. 169

CHAPTER 7. OPTIMIZATION 20. Suppose a consumer maximizes her utility by buying two products with amounts x1 and x2 according to the utility function U (x1 , x2 ) = xα1 x21−α . Her budget constraint is p1 x1 + p2 x2 = y where p1 , p2 are the prices and y is her weekly expenditure on the two products. (a) Find the weekly ordinary demand function for the two products. (b) Define the indirect utility function V (p1 , p2 , y) = U (x∗1 , x∗2 ) to be the value function of the optimal choice. Show that ∂V (p1 , p2 , y) = λ, ∂y where λ is the Lagrange multiplier. 21. Find the maximum of the function f (x, y) = x2 −y 2 in the constraint set {(x, y) ∈ R2 : x2 + y 2 ≤ 1}. 22. Solve the following maximization problem: max 2 log x + y x,y

subject to x + 2y ≤ 1, x ≥ 0, y ≥ 0. 23. The utility function of a consumer is given by U (x1 , x2 ) = (x1 x2 )1/2 , where prices for goods 1 and 2 are p1 > 0 and p2 > 0. Given that the consumer has income y > 0, solve the utility maximization problem subject to the constraints, p1 x1 + p2 x2 ≤ y, x1 ≥ 0, x2 ≥ 0 as follows: (a) Show that this can be reduced to an equality-constrained problem. (b) Find the critical point(s) of the Lagrangian. (c) Check the second-order conditions. 24. Suppose that a consumer’s preferences are represented by an increasing and quasiconcave utility function C 2 function u = f (x), where x ∈ Rn+ is the consumption bundle. The budget constraint is given by pT x = y, where p ∈ Rn++ is the price vector and y > 0 income. (a) Set up the utility maximization problem as a equality-constrained optimization problem. 170

7.4. EXERCISE (b) Let x∗ be the optimal consumption bundle and V (p, y) be the value function (called the indirect utility function). Use the envelope theorem to prove Roy’s identity: ∂V (p, y)/∂pi x∗i = − , i = 1, . . . , n ∂V (p, y)/∂y 25. A competitive firm produces a single output Q with capital K and labour L with production technology according to an increasing and strictly concave production function Q = F (L, K). The market price of the output is p and the market prices of capital and labour are r and w respectively. (a) Set up the profit maximization problem as an equality-constrained maximum problem. Identify the control variables. (b) Set up the Lagrangian. (c) The value function, called the profit function is written as π(p, w, r). Prove the Hotelling’s lemma: ∂π(p, w, r) = Q∗ , ∂p where Q∗ denotes the optimal level of output. 26. Use the Kuhn-Tucker Theorem to solve the following optimization problem: max [−(x − 4)2 − (y − 4)2 ] x,y

subject to x + y ≤ 4, x + 3y ≤ 9. 27. A consumer who buys two goods has a utility function U (x1 , x2 ) = min{x1 , x2 }. Given income y > 0 and prices p1 > 0 and p2 > 0. (a) Describe the consumer’s utility maximization problem. (b) Does the Weierstrass theorem apply to this problem? (c) Can the Kuhn-Tucker theorem be used to obtain a solution? (d) Solve the maximization problem. Explain your answers. 171

CHAPTER 7. OPTIMIZATION 28. Solve the following maximization problem: Maximize xy subject to x + 2y ≤ 5, x ≥ 0, y ≥ 0. 29. Solve the following maximization problem: Maximize x + log(1 + y) subject to x + 2y = 1, x ≥ 0, y ≥ 0. 30. An inequality-constrained optimization problem is given by max x2 − y 2 x,y

subject to x2 + y 2 ≤ 4, y ≥ 0, y ≤ x. (a) Derive the Kuhn-Tucker conditions by assuming that only the first constraint is binding. Determine whether a solution exists with this assumption. (b) Find the solution of the problem.

172

Chapter 8 Probability Most graduate level econometrics textbooks require the readers to have some background in intermediate level mathematical probability and statistics. A lot of students, however, took business and economic statistics and then econometric courses in the undergraduate program. This chapter gives a quick introduction to the formal theory. It is by no mean a replacement for the proper training. Knowledge in probability theory is also essential in the studies of economics of risk and uncertainty, financial economics, game theory, and macroeconomics.

8.1

Basic Definitions

Throughout history people involved in decision making and gambling activities have developed an intuitive idea of probability, which is the likeliness of some events would happen. The formal theory was first constructed by Blaise Pascal and Pierre de Fermat in the seventeenth century (see Ross, 2004). We start with the idea of a random experiment or random trial. Before the experiment, it is assumed that we know the set of all possible outcomes, which is called the sample space, denoted by Ω. At the end of the experiment, one outcome is realized.

Example 8.1. A coin is tossed once, there are two possible outcomes: 173

CHAPTER 8. PROBABILITY

Let us call the outcome on the left “head”, or simply H, and the one on right “tail”, or T . Then the random experiment is the tossing of the coin, the sample space is Ω = {H, T }. At the end of the experiment either H or T is the outcome, but not both. Example 8.2. A die is thrown once. The sample space Ω consists of six outcomes:  u  

r  r

r r  r

r r r r 

r r r r r 

r r rr r r 

Often we want to represent the outcomes by numbers. So we define a function X : Ω → R called a random variable. Random variables are usually denoted by capital letters, say, X. In example 8.1, we can define X(H) = 0,

X(T ) = 1,

so that the range of the random variable X is {0, 1}. In example 8.2, the natural way to assign values to the variable is by the number of dots on the face of the die. In this case the range of X is {1, 2, 3, 4, 5, 6}. It is clear that a random variable is everywhere defined but not necessary one-to-one or onto. Nevertheless, the assignment of a random variable to the sample space may not be meaningful, as the following example shows. Example 8.3. A card is chosen randomly from a deck of cards. The sample space is Ω = {♠A, ♠2, . . . , ♠10, ♠J, ♠Q, ♠K, ♥A, . . . , ♥K, ♦A, . . . , ♦K, ♣A, . . . , ♣K}, with 52 possible outcomes. The sample space can also be seen as a product set 174

8.1. BASIC DEFINITIONS Ω = S × T , where S = {♠, ♥, ♦, ♣}, T = {A, 2, . . . , 10, J, Q, K}. We shall see later in example 8.11 that independent events can be constructed from the product sets. A subset of the sample space is called an event. Since events are sets, the set operations defined in chapter 2 can be used to defined new events. For example, the event A ∪ B means that the outcome belongs to A or to B. Two events A ad B are said to be mutually exclusive if A and B are disjoint. The complement of A, Ac is the event that A does not happen. Example 8.4. Let A be the event that the randomly chosen card be a seven. Then A = {♠7, ♥7, ♦7, ♣7}. Let B be the event that the card is a heart, that is, B = {♥A, . . . , ♥K}. Then the event A \ B is the event of a seven card but not ♥7. That is, A \ B = {♠7, ♦7, ♣7}. Example 8.5. In example 8.1 a coin is tossed once, with Ω = {H, T }. Suppose now that the coin is tossed twice. The sample space is Ω2 = {(H, H), (H, T ), (T, H), (T, T )}. Let A be the event that we have two heads, B means two tails, and C means one head and one tail. Then A, B, C is a partition of the sample space Ω. In other words, the events are pairwise mutually exclusive and A ∪ B ∪ C = Ω. The likeliness that an event will happen is represented by a number between 0 and 1 assigned to the event. Formally, a probability measure is a function P which maps the set of all events of a sample space Ω into [0, 1] which satisfies the following axioms: 1. P (A) ≥ 0 for every A ⊆ Ω. 2. P (Ω) = 1. 3. If A and B are mutually exclusive, then P (A ∪ B) = P (A) + P (B). 175

CHAPTER 8. PROBABILITY In assigning probabilities to events, we often take the frequentist approach or the subjective approach. In the first approach, we imagine that we can repeat the same random experiment many times, say N and observe the number of times n that an event A occurs. Then we assume that P (A) = n/N as N → ∞. In the subjective approach, the analyst assigns the probability according to her subjective opinion or based on some deductive reasonings. In simple cases the two approaches can give the same result. For example, if the sample space Ω is finite, and each outcome is considered equally likely to occur, then the probability of an event A can be defined as |A| . (8.1) P (A) = |Ω| That is, the likeliness of event A happens is the ratio of the number of outcomes in A and the size of the sample space. Example 8.6. Suppose a coin is tossed once. If we assume that the coin is fair, there is no reason to believe that the outcome H is more likely than T , or vice versa. Therefore it is natural to assign the probabilities P ({H}) = P ({T }) = 1/2. Example 8.7. Consider the experiment in example 8.5. Again if we assume a fair coin, the probabilities of the four outcomes are the same. Then P (A) = P (B) = 1/4. By the axioms above P (C) = 1/2. Example 8.8. It is often more convenient to define the probabilities on the random variable instead of events. Consider the experiment in throwing a fair die in example 8.2. It is natural to assign equal probability of 1/6 to each outcome in Ω. Therefore P (X = 1) = P (X = 2) = · · · = P (X = 6) = 1/6. The probability of the event that X is less than or equal to 2 can be written as P (X ≤ 2) = P (X = 1) + P (X = 2) = 1/3. Similarly, P (X is odd) = P (X = 1) + P (X = 3) + P (X = 5) = 1/2. The following results can be proven directly from the three axioms of probability 176

8.2. CONDITIONAL PROBABILITY and left as an exercise. Theorem 8.1. Let A and B be events of a sample space Ω. Then 1. P (Ac ) = 1 − P (A). 2. If A ⊆ B, then P (A) ≤ P (B). 3. P (A ∪ B) = P (A) + P (B) − P (A ∩ B). 4. P (A \ B) = P (A) − P (A ∩ B). 5. P (∅) = 0. When the sample space Ω is a finite or countable set, the set of all events is the power set of Ω. The probability function defined above is well-defined. Things are a bit more complicated if Ω is uncountable. In order for the probability function to be well-defined, we often narrow the set of events to a collection of sets called a σ-algebra or σ-field, which is closed under countable unions, intersections, and complements. Formally, a σ-field F is a collection of subsets of the sample space Ω which satisfies the following three axioms: 1. ∅ ∈ F, 2. For every A ∈ F, the complement, Ac ∈ F, 3. If {A, B, C, . . . } are in F, then A ∪ B ∪ C ∪ · · · is also in F. By axioms 1 and 2, the sample space Ω is always in F. Also, axioms 2 and 3 and De Morgan’s Law implies that any countable intersection of events in F is also in F.

8.2

Conditional Probability

Suppose that we are concerned with the occurrence of a particular event A within a sample space Ω. Before the outcome is revealed, we are given an additional piece of information, that event B has occurred. Will this information change our perspective on the probability of the event A? Prior to the random experiment, suppose we know that the probabilities of event A is P (A). We also know P (B), and P (A ∩ B). Now given that we know event B has been realized, is the probability of A still P (A)? Formally, we are evaluating the conditional probability of event A given that event B has occurred, which is denoted by P (A|B). Consider the following two special cases: 177

CHAPTER 8. PROBABILITY 1. Suppose that A and B are mutually exclusive. If B is realized, then A cannot happen. In this case P (A|B) = 0. 2. Suppose that B ⊆ A. By definition B is realized means that A is also realized so that P (A|B) = 1. In the general case, it depends on P (A ∩ B). Since we know that B has occurred, the sample space is no longer Ω but B. Therefore we have to “rescale” P (A ∩ B) using B as the sample space so that all the mutually exclusive events still sum to 1. It follows that P (A ∩ B) P (Ω) = . (8.2) P (A|B) = P (A ∩ B) × P (B) P (B) Notice that the two special cases above satisfy equation (8.2) as well. In the first case, P (A ∩ B) = 0 and in the second P (A ∩ B) = P (B). Also, since event B is realized, P (B) 6= 0 so that equation (8.2) is well-defined. Example 8.9. Consider the probability of getting the queen of heart from a random draw from a desk of cards, which is P (A) = 1/52. Before the card is revealed, we are told that the card is a heart. So it is clear that from the new information the probability of getting the queen of heart is 1/13. Formally, A is the event of getting the queen of heart. B is the event of getting a heart, with P (B) = 1/4. In this case A ⊆ B so that P (A ∩ B) = P (A) = 1/52. It follows that P (A|B) =

1/52 1 P (A ∩ B) = = . P (B) 1/4 13

Psychologists Amos Tversky and Daniel Kanheman (1974) point out that intuition is very unreliable when people make subjective probabilities. The following example, taken from Economist (1999), illustrates the importance of the concept of conditional probability. Most people, unfortunately probably including your family doctor, get the answer wrong the first time. Example 8.10. You are given the following information. (i) In random testing, you test positive for a disease. (ii) In 5% of cases, this test shows positive even when the subject does not have the disease. (iii) In the population at large, one person in 1,000 has the disease. What is the probability that you have the disease? Let A be the event that you have the disease, and B be the event that you test positive for the disease. So P (A) = 0.001 and P (B) = 0.05. We can assume that if you have the disease you will be tested positive as well. Therefore, A ⊆ B so that 178

8.2. CONDITIONAL PROBABILITY P (A ∩ B) = P (A) = 0.001. It follows that P (A|B) =

0.001 P (A ∩ B) = = 0.02. P (B) 0.05

That is, even though you test positive, the probability of having the disease is only 2%. As the article points out, most people’s answers is 95%. They fail to include the information in (iii) in their subjective probabilities. Suppose that A and B are two events from a sample space Ω. We say that A is independent of B if P (A|B) = P (A). That is, the probability of event A is unaffected by whether event B happens or not. Using equation (8.2), we have P (A|B) =

P (A ∩ B) = P (A), P (B)

which gives P (A ∩ B) = P (A)P (B).

(8.3)

It can be shown easily that if A is independent of B, then B is also independent of A so that the relation is symmetric. Equation (8.3) says that if two events are independent, then the probability that they both occur is the product of their probabilities. Example 8.11. Let the event A be drawing a 7 from a deck of cards and B be drawing a diamond. Then P (A) = 1/13, P (B) = 1/4, and P (A ∩ B) = 1/52. It follows that P (A ∩ B) 1/52 1 = = = P (A), P (B) 1/4 13 P (A ∩ B) 1/52 1 P (B|A) = = = = P (B), P (A) 1/13 4 P (A|B) =

so that A and B are independent. Independent events arise from situations when the sample spaces are product sets, as examples 8.3 and 8.11 show. A common situation is when we repeat a random experiment n times, each with a sample space Ω. The sample space of all the n trials is Ωn . Then any event in one particular trial is independent of any event in another trial. For example, if we roll a die ten times, even the outcomes of the first nine are all even, the probability of the last roll being odd is still one half. 179

CHAPTER 8. PROBABILITY By the definition of conditional probability, we have P (B|A) =

P (A ∩ B) , P (A)

which gives P (A ∩ B) = P (B|A)P (A).

(8.4)

Substitute equation (8.4) into (8.2), we obtain Bayes’ Theorem: P (A|B) =

P (B|A)P (A) . P (B)

There are many interpretations of Bayes’ Theorem. Here we provide one in line with the question we ask at the beginning of the section. Suppose we have a prior believe that the probability of event A happens is P (A). Then we learn that another event B has occurred. Given the new information, how would we update our believe on event A? Bayes’ Theorem says that we should update P (A) by multiplying it with the factor P (B|A)/P (B), the ratio of the conditional probability of B given A and the probability of B.

8.3

Probability Distributions

Consider a random experiment with a countable sample space Ω. Then the range of the random variable X is a countable set of real numbers with the same cardinality as Ω, which can be denoted by S = {x1 , x2 , x3 , . . . }. The set S is called the support of the random variable X. We define the probability mass function f : S → [0, 1] as f (x) = P (X = x). That is, f (x) is the probability of the random variable X equal to x. Example 8.12. Consider the experiment of rolling a fair die once. The support of the random variable X is S = {1, 2, 3, 4, 5, 6}. The probability mass function is  1/6 x = 1, 2, . . . , 6, f (x) = 0 elsewhere. By the axioms of probability, the probability mass function (pmf) satisfies the 180

8.3. PROBABILITY DISTRIBUTIONS following two properties: 1. f (x) ≥ 0 for all x ∈ R, P 2. x∈S f (x) = 1. The cumulative distribution function, or simply distribution function F : S → [0, 1] is defined as F (x) = P (X ≤ x). (8.5) It is clear that F is an increasing function of x, and F (x) =

X

f (X).

X≤x

Example 8.13. The distribution function in example 8.12 is

F (x) =

   0

x < 1,

  

x ≥ 6.

n/6 n ≤ x < n + 1, n = 1, 2, 3, 4, 5, 1

When the sample space Ω is finite or countable, X is called a discrete random variable. Now we consider the case when Ω is uncountable, which is often the case when we measure a quantity such as time, price, quantity, inflation, expenditure, utility, etc. Events are often described by intervals on the real line. In this case we call X a continuous random variable. The distribution function of X is the same as defined in equation 8.5. The distribution function of a continuous random variable is increasing and also continuous. Unlike the discrete case, there is no x such that P (X = x) has a positive value, that is, P (X = x) = 0 for all x ∈ R. In many cases, the distribution function of a continuous random variable can be expressed as Z x F (x) = f (t) dt, −∞

where f is called the probability density function (pdf) of X. By the Fundamental Theorem of Calculus, F 0 (x) = f (x) on the support of X, which is defined as S = {x ∈ R : f (x) > 0}. 181

CHAPTER 8. PROBABILITY From the above definition of F , the probability of an event A = [a, b] is b

Z P (a ≤ X ≤ b) = F (b) − F (a) =

f (t) dt. a

By the axioms of probability, f is required to satisfy the following properties: 1. f (x) ≥ 0 for all x ∈ R, 2.

R∞ −∞

f (t) dt = 1.

Example 8.14. A random experiment involves choosing a point X in the interval [0, 1]. (Most electronic calculators have a function generating random numbers.) The distribution function of X is    0 x < 0, F (x) =

x 0 ≤ x ≤ 1,

  

1

x > 1.

The pdf of X is  1 0 ≤ x ≤ 1, f (x) = 0 elsewhere. The probability that the number is less than 1/2 is Z

1/2

P (X < 1/2) = F (1/2) =

Z f (t) dt =

−∞

0

1/2

1 1 dt = . 2

Other similar distributions, with the pdf equal to a constant value on an interval, is generally called uniform distributions.

8.4

Mathematical Expectations

Suppose that X is a random variable with a pmf or pdf f (x). Let u(X) be a function of X. Suppose further that, in the discrete case, X

|u(x)|f (x) ≤ ∞,

x∈S

182

(8.6)

8.4. MATHEMATICAL EXPECTATIONS or in the continuous case, Z



|u(x)|f (x) dx ≤ ∞,

(8.7)

−∞

then we can define the mathematical expectation of u(X) in the discrete case as X

E[u(X)] =

u(x)f (x),

x∈S

and in the continuous case, ∞

Z

u(x)f (x) dx.

E[u(X)] = −∞

Given a random variable X, let V (X) be the set of functions of X that satisfy inequality (8.6) in the discrete case and inequality (8.7) in the continuous case. Then it can be shown that V (X) is a vector space. By the properties of integration, mathematical expectation is a linear functional on V (X). That is, for any u, v ∈ V (X) and α ∈ R, E[αu(X) + v(X)] = αE[u(X)] + E[v(X)].

(8.8)

Some of the important expectations for the continuous case are discussed as follows. The counterparts for the discrete case are similar by using summation instead of integration. If u(X) = X, then Z ∞ E(X) = xf (x) dx −∞

is called the expected value of X or simply the mean of X, and sometimes denoted by µ. Intuitively, it gives the notion of the average of the outcomes if we repeat the random experiment many times. The concept is also analogous to the centre of gravity of a rigid body in physics. For this reason the mean is sometimes called the first moment of the probability distribution about the point zero. If u(X) = (X − µ)2 , then 2

Z



E[(X − µ) ] =

(x − µ)2 f (x) dx

−∞

is called the variance of X and is denoted by σ 2 or Var(X). It takes the average 183

CHAPTER 8. PROBABILITY of the square of the distance of each of the outcomes weighted by the probabilities. Therefore σ 2 is a measure of how spread out the outcomes are away from the mean. For this reason it is sometimes called the second moment of the distribution about the mean. The positive square root of the variance, σ, is called the standard derivation of X. Some statisticians prefer the use of a dimensionless quantity to measure the dispersion, which is called the coefficient of variation and is defined as σ cv = , µ provided that µ 6= 0. By the linear property described in equation (8.8), we have E[(X − µ)2 ] = E(X 2 − 2µX + µ2 ) = E(X 2 ) − 2µE(X) + µ2 = E(X 2 ) − µ2 , which is sometimes a convenient way to calculate σ 2 . Example 8.15. Consider again the experiment of rolling a fair die once. The pmf is given in example 8.12. The mean is µ = E(X) =

6 X 1 x=1

The variance is σ 2 = E[(X − µ)2 ] =

6 X 1 x=1

The standard derivation is σ = cv = 1.708/3.5 = 0.488.



6

6

x = 3.5.

(x − µ)2 = 2.917.

2.917 = 1.708. The coefficient of variation is

Example 8.16. Consider a random variable X with a uniform pdf  1/(2a) −a < x < a, f (x) = 0 elsewhere. We have

Z

a

µ = E(X) = −a

184

1 x dx = 0, 2a

8.4. MATHEMATICAL EXPECTATIONS and 2

Z

2

a

σ = E[(X − µ) ] = −a

1 2 a2 x dx = . 2a 3

√ This gives σ = a/ 3. The coefficient of variation cv is undefined since µ = 0. Example 8.17. Let us repeat example 8.16 with the pdf  1/(4a) −2a < x < 2a, f (x) = 0 elsewhere. √ In this case µ = 0 and σ = 2a/ 3. The standard derivation is two times that of the previous example since the distribution is more spread out. The following results involve simple functions of a random variable. The proofs are left as exercises. Theorem 8.2. Suppose that X is a random variable with mean µ and variance σ 2 . 1. Let Y = a + bX, where a, b ∈ R and b 6= 0. Then E(Y ) = a + bµ,

Var(Y ) = b2 σ 2 .

2. Let

X −µ . σ Then Z is called the standardized random variable of X and has mean equals to 0 and variance 1. Z=

The third moment of a standardized random variable X is called the coefficient of skewness, cs , and is a measure of the symmetry of the distribution about the mean. Formally, E[(X − µ)3 ] cs = . σ3 The coefficient is negative when the distribution of X is skewed to the right, positive when skewed to the left, and zero when the distribution is symmetric about the mean. It is straight forward to show that cs =

E(X 3 ) − 3µσ 2 − µ3 . σ3 185

(8.9)

CHAPTER 8. PROBABILITY The fourth moment of the standardized random variable is defined as the coefficient of kurtosis, which is ck =

E[(X − µ)4 ] . σ4

There are different interpretations of ck . The most common one is that it is a measure of the weight of the tails of the distribution. A large coefficient means that X has fat tails. The concept is similar to the moment of inertia of beams and columns in structural mechanics.1

8.5 8.5.1

Some Common Probability Distributions Binomial Distribution

Suppose that a random experiment is repeated n times. In each trial the probability of an event occurring is p. Let X be the random variable of the number of trials that the event occurs. Then the support of X is S = {0, 1, 2, . . . , n}. The probability mass function of X is the binomial distribution ! n x f (x) = p (1 − p)n−x , x = 0, 1, 2, . . . , n, x where

! n! n = x!(n − x)! x

is the number of combinations of taking x outcomes from a sample space of size n without repetitions. The binomial distribution has mean µ = np and variance σ 2 = np(1 − p). Example 8.18. A fair die is rolled 6 times. Let A be the event that the outcome is either 1 or 2 each time. Let X be the number of times that event A occurs. Then we have n = 6 and p = 1/3. The values of the pmf of X are listed below: x f (x)

0 0.088

1 0.263

2 0.329

3 0.219

4 0.082

5 0.016

6 0.001

The mean of the distribution is µ = 2 and the variance σ 2 = 4/3. The readers can P verify that 6x=0 f (x) = 1. 1

See, for example, Timoshenko (1940, p. 343).

186

8.5. SOME COMMON PROBABILITY DISTRIBUTIONS

8.5.2

Geometric Distribution

Suppose a random experiment is repeated until a certain event A with probability p occurs. Let X be the number of trials until A first happens. Then the support of X is N, the set of natural numbers. The pmf of X is f (x) = p(1 − p)x ,

x = 1, 2, 3, . . . .

The distribution has mean µ = 1/p and variance σ 2 = (1 − p)/p2 . Example 8.19. A fair die is rolled until the event A = {1, 2} occurs. Let X be the number of trials that A first happens. Then p = 1/3. The values of the pmf of the first six trial are as follows. x f (x) F (x)

1 0.333 0.333

2 0.222 0.556

3 0.148 0.704

4 0.099 0.802

5 0.066 0.868

6 0.044 0.912

The mean of the distribution is µ = 3 and the variance σ 2 = 6. It is interesting to point out that although the probability of A in each of the independent trial is 1/3, the cumulative probability of X = 3 is only 0.704. In fact, the cumulative distribution function of a geometric distribution is F (x) = 1 − (1 − p)x ,

8.5.3

x = 1, 2, 3, . . . .

Gamma and Chi-Square Distributions

For 0 < x < ∞, define the gamma function as2 Z ∞ Γ(x) = e−t tx−1 dt. 0

Using integration by parts, it can be shown that Γ(x + 1) = xΓ(x). It follows that for a positive integer n, Γ(n + 1) = n!. 2

See Artin (1964) for the properties of the gamma function.

187

CHAPTER 8. PROBABILITY Also,



Z

e−t dt = 1,

Γ(1) = 0

so that we define 0! = 1 as before. A continuous random variable X has a gamma distribution if the pdf has the form f (x) =

1 xα−1 e−x/β , Γ(α)β α

0 < x < ∞,

where α > 0 and β > 0 are two parameters. The mean of the distribution is µ = αβ and the variance is σ 2 = αβ 2 . The gamma distribution is a very good model of income distribution because of the flexibility provided by the two parameters. A special case of the gamma distribution is when α = k/2 and β = 2. The pdf becomes 1 f (x) = xk/2−1 e−x/2 , 0 < x < ∞. Γ(k/2)2k/2 It has a special name called a chi-square distribution with k degrees of freedom, and is denoted by χ2 (k). The distribution has mean µ = k and variance σ 2 = 2k. The χ2 distribution has a close relationship with the normal distribution, which we shall define next.

8.5.4

Normal Distribution

The normal distribution is the most important distribution in probability theory because of the central limit theorem. The general form of the pdf has two parameters µ and σ, and is given by "  2 # 1 x−µ 1 exp − , −∞ < x < ∞. f (x) = √ 2 σ 2πσ The normal distribution is often denoted by N (µ, σ 2 ). The mean of the distribution is µ and variance σ 2 . In practice, it is often convenient to transform the distribution it standardized form, with X −µ Z= . σ By theorem 8.2, Z has mean 0 and variance 1, with pdf  2 1 z f (z) = √ exp − , −∞ < z < ∞. 2 2π 188

8.5. SOME COMMON PROBABILITY DISTRIBUTIONS y

x −3

−2

−1

0

1

2

3

Figure 8.1: Standard Normal Distribution

The normal distribution is symmetric about its mean and therefore the coefficient of skewness cs is zero. Because of the shape of the graph, it is often call the bell curve (see figure 8.1). The following two results are very useful in statistical analysis. Theorem 8.3. Let X be a normally distributed random variable with mean µ and variance σ 2 . Let Z = (X − µ)/σ be the standardized random variable of X. Then Y = Z 2 is χ2 (1). ¯ n = (X1 + X2 + · · · + Xn )/n be the mean of n independent Theorem 8.4. Let X random samples of size n from a normal distribution that has mean µ and variance ¯ n is N (µ, σ 2 /n). σ 2 . Then X The central limit theorem was presented in chapter 3 as an example of functional convergence. We restate the theorem here in a slightly different form. Theorem 8.5 (Central Limit Theorem). Suppose that {X1 , X2 , . . . , Xn } is a sequence of outcomes from n independent random trials from a distribution with mean µ and variance σ 2 . Then the random variable √ ¯ n(Xn − µ) . Zn = σ ¯ n = (X1 + X2 + · · · + Xn )/n converges in distribution to a random variable where X which has a standard normal distribution N (0, 1). 189

CHAPTER 8. PROBABILITY

8.6

Multivariate Distributions

When we have two random variables, say X1 and X2 , we consider the joint distribution with pdf f : R2 → R+ where Z ∞Z ∞ f (x1 , x2 ) dx1 dx2 = 1. −∞

−∞

The marginal distribution of X1 is defined by the pdf Z ∞ f1 (x1 ) = f (x1 , x2 ) dx2 . −∞

The marginal distribution of X2 can be defined similarly. The means and variances of X1 and X2 are then defined by their marginal pdf. Mathematical expectation is a linear operator. That is, for any scalar α, E[αX1 + X2 ] = αE[X1 ] + E[X2 ]. In addition to their means, µ1 and µ2 , and variances, σ12 and σ22 , we need to ask whether they are correlated or not. The correlation is expressed by their covariance, σ12 = E[(X1 − µ1 )(X2 − µ2 )] = E(X1 X2 − µ2 X1 − µ1 X2 + µ1 µ2 ) = E(X1 X2 ) − µ2 E(X1 ) − µ1 E(X2 ) + µ1 µ2 = E(X1 X2 ) − µ1 µ2 This is often expressed in the form E(X1 X2 ) = E(X1 )E(X2 ) + Cov(X1 , X2 ),

(8.10)

where Cov(X1 , X2 ) = σ12 . The conditional distribution of X2 given the value of X1 = x1 is defined as f2|1 (x2 |x1 ) = 190

f (x1 , x2 ) . f1 (x1 )

(8.11)

8.6. MULTIVARIATE DISTRIBUTIONS The conditional mean or expectation of X2 given X1 = x1 is Z ∞ x2 f2|1 (x2 |x1 ) dx2 . E[X2 |x1 ] = −∞

We say that the two random variables are independent if their joint pdf is separable, that is, f (x1 , x2 ) = f1 (x1 )f2 (x2 ). This implies that their covariance is zero since x σ12 = (x1 − µ1 )(x2 − µ2 )f (x1 , x2 ) dx1 dx2 x = (x1 − µ1 )(x2 − µ2 )f1 (x1 )f2 (x2 ) dx1 dx2 Z Z = (x1 − µ1 )f1 (x1 ) dx1 (x2 − µ2 )f2 (x2 ) dx2 = 0. It follows that (8.10) becomes E(X1 X2 ) = E(X1 )E(X2 ). It is also clear from (8.11) that if X1 and X2 are statistically independent then f2|1 (x2 |x1 ) = f2 (x2 ), that is, the conditional distribution of X2 is independent of the value of X1 . All these are conveniently expressed in matrix form as follows: ! ! ! X1 µ1 σ12 σ12 X= , E(X) = , Σ= , X2 µ2 σ21 σ22 where Σ is a symmetric matrix called the variance-covariance matrix (or simply called the covariance matrix). This idea of course can be extended to a general nvector. When X is a n-tuple of random variables, E(X) = µ is a n-vector of means and Σ = E(X − µ)(X − µ)T is a n × n symmetric covariance matirx, where the i-jth element is σij . It can be readily shown that Σ is positive definite. Also, if c ∈ Rn is a constant vector, then E(cT X) = cT E(X) = cT µ 191

CHAPTER 8. PROBABILITY and Var(cT X) = cT Σc. The most frequently joint distribution we encounter is the multivariate normal distribution, with joint pdf   1 1 T −1 f (x) = exp − (x − µ) Σ (x − µ) . (2π)n/2 |Σ|1/2 2 The following fact is often used in econometric analysis: If X is a random n-vector which has a multivariate normal distribution, with mean µ and covariance matrix Σ, then the statistic (X − µ)T Σ−1 (X − µ) has a χ2 distribution with degree n.

In stochastic economic models we frequently employ the assumption of certainty equivalence. That is, we use the conditional mean of a random variable in an optimization problem. Care should be taken in applying the statistical rules. For example, expected utility is different from utility of the expected value, that is, E[u(X)] 6= u(E[X]). Similarly, 

 1 1 E . 6 = X E[X] In these cases the functions are linearized for further analysis. For example, let the discounted value of consumption in period t + 1 be βct+1 where 0 < β < 1 is the discount factor. Using the first-order Taylor approximation at c∗ , the steady-state consumption level,      1 1 1 1 ∗ Et − (ct+1 − c ) ' Et βct+1 β c∗ (c∗ )2   1 Et [ct+1 ] = 2− . ∗ βc c∗ Also, in general E(X1 X2 ) 6= E(X1 )E(X2 ) unless X1 and X2 are statistically independent. 192

8.7. STOCHASTIC PROCESSES

8.7

Stochastic Processes

A stochastic process is a sequence of random variables, {Xt }, t = 0, 1, . . . . In this section we introduce several useful models in economics.

8.7.1

Martingales

A stochastic process is called a martingale if for t = 0, 1, . . . , 1. E[|Xt |] < ∞, 2. E[Xt+1 |X0 , X1 , . . . , Xt ] = Xt . Taking expectation on both sides of condition 2 above gives E[Xt+1 ] = E[Xt ]. Therefore a martingale has constant mean. If all the Xt have independent and identical distributions, then the process is called a random walk.

8.7.2

Moving Average Models

A sequence of random variables {t } is called a white noise process if for all t 6= s, 1. E[t ] = 0, 2. E[2t ] = σ 2 , 3. Cov(t , s ) = 0. A first-order moving average process, MA(1), is defined as Xt = µ + t + θt−1 , where µ and θ are constants. The expected value of Xt in each period is E[Xt ] = E[µ + t + θt−1 ] = µ + E[t ] + θE[t−1 ] = µ. 193

CHAPTER 8. PROBABILITY The variance is E(Xt − µ)2 = E(t + θt−1 )2 = E(2t + 2θt t−1 + θ2 2t−1 ) = σ 2 + 0 + θ2 σ 2 = (1 + θ2 )σ 2 . In general, a moving average process of order q, MA(q), is defined as Xt = µ + t + θ1 t−1 + θ2 t−2 + · · · + θq t−q .

8.7.3

(8.12)

Autoregressive Models

A first-order autoregressive process, AR(1), is defined as Xt = c + φXt−1 + t . This is in fact a first-order difference equation with non-constant coefficients as specified in equation (9.3), with a = φ and bt−1 = c + t . If −1 < φ < 1, we can solve the equation backward using equation (9.5), that is, Xt =

∞ X

φs−1 (c + t−s+1 )

s=1 ∞

X c = φs t−s . + 1 − φ s=0

(8.13)

Finding the mean and the variance of this AR(1) process is left as an exercise. In this case the process has a constant mean and variance and is said to be stationary. On the other hand, if φ < −1 or φ > 1, the process is non-stationary. Comparing equations (8.12) and (8.13) reveals that an AR(1) process can be expressed as a MA(∞) process with µ = c/(1 − φ) and θi = φi . In general a stationary AR process of any order can be inverted into a MA process and vice versa. It is often advantageous to use a mixed autoregressive and moving average model. For example, an ARMA(1, 1) process is defined as Xt = c + φXt−1 + t + θt−1 . 194

8.8. EXERCISES More on stochastic processes can be found in Taylor and Karlin (1998), Box, Jenkins, and Reinsel (1994), and Hamilton (1994). See Grinstead and Snell (1997), Hogg and Craig (1995), or any book in mathematical probability and statistics for more detailed discussions on probability theory.

8.8

Exercises

1. In a random experiment, a coin is flipped until the first tail appear. Describe the sample space of the experiment. 2. In the game of craps, two dice are thrown and the sum of the number of dots on the dice are recorded. Describe the sample space of each throw. 3. Prove theorem 8.1. 4. Show that the probability function defined in equation (8.1) satisfies the three axioms of probability measure. 5. Consider the experiment of drawing a card randomly from a deck of well-shuffled cards. (a) What is the probability of the outcome is ♣10? (b) Find the probabilities of events A, B, and A \ B defined in example 8.4. 6. Let Ω = {a, b, c, d}. Determine if the following collections are σ-fields. (a) F = {∅, Ω} (b) F = {∅, {a}, {d}, {a, d}, {b, c}, Ω} (c) F = {∅, {a, b}, {c, d}, Ω} 7. Show that the following collections are σ-fields of a sample space Ω: (a) {∅, Ω}. (b) {∅, A, Ac , Ω}, for any A ⊆ Ω. (c) P(Ω), the power set of Ω. 8. Show that if event A is independent of event B, then B is also independent of A. 195

CHAPTER 8. PROBABILITY 9. (Wiggins, 2006) A patient goes to see a doctor. The doctor performs a test with 99 percent reliability—that is, 99 percent of people who are sick test positive and 99 percent of the healthy people test negative. The doctor knows that only 1 percent of the people in the country are sick. If the patient tests positive, what are the chances the patient is sick? 10. (Paulos, 2011) Assume that you’re presented with three coins, two of them fair and the other a counterfeit that always lands heads. If you randomly pick one of the three coins, the probability that it’s the counterfeit is 1 in 3. This is the prior probability of the hypothesis that the coin is counterfeit. Now after picking the coin, you flip it three times and observe that it lands heads each time. Seeing this new evidence that your chosen coin has landed heads three times in a row, what is the revised posterior probability that it is the counterfeit? 11. Suppose that {B1 , B2 , . . . , Bn } is a partition of a sample space Ω. Prove the following law of total probabilities: For any event A ⊆ Ω, P (A) =

n X

P (A|Bi )P (Bi ).

i=1

12. Consider example 8.5 of tossing a fair coin twice. Let X be the random variable of the number of times that tail shows up. (a) What is the support of X? (b) What is the probability mass function of X? 13. Prove equation (8.8). 14. Find the mean, variance, and coefficient of variation of the random variable defined in example 8.14. 15. Prove theorem 8.2. 16. Prove equation (8.9). 17. The price of a new iTab computer is $500. It is known that a fraction d of the iTabs are defective. All consumers value the non-defective computers at $550 each. The same model of second-hand iTab can be found on eBay for $50, which includes shipping costs. 196

8.8. EXERCISES (a) Do you expect to find non-defective iTabs for sale on eBay? Why or why not? (b) If the consumers are risk neutral, what is d? 18. A fair die is rolled 6 times. Let A be the event that the outcome is 5 each time. Let X be the number of times that event A occurs. Find the pmf, the mean, and the variance of X. 19. Show that

Z



f2|1 (x2 |x1 ) dx2 = 1. −∞

20. Let X1 and X2 have the joint pdf ( f (x1 , x2 ) =

2, 0 < x1 < x2 < 1, 0

elsewhere.

(a) Find the marginal pdf of X1 and X2 . (b) Find the conditional pdf of X1 given X2 = x2 . (c) Find the conditional mean of X1 given X2 = x2 . 21. Mr. Smith has two children, at least one of whom is a boy. What is the probability that the other is a boy? (Confused? See the discussion in Wikipedia.) 22. Consider the stock price model in section 9.1.2. Suppose the dividends of the stock follow a random walk, that is, dt = dt−1 + et , where et has an independent and identical distribution in every period with E[et ] = 0. Instead of the adaptive expectation as in equation (9.7), the investors have rational expectations, Et [pt+1 ] = pt+1 . (a) Derive the stock price pt . (b) Repeat the analysis if the dividends are constant, that is, dt = d for all t. 23. Find the mean and variance of the AR(1) process if −1 < φ < 1. 24. Consider the following passage from Grinstead and Snell (1997, p. 405): 197

CHAPTER 8. PROBABILITY Most of our study of probability has dealt with independent trials processes. These processes are the basis of classical probability theory and much of statistics. We have discussed two of the principal theorems for these processes: the Law of Large Numbers and the Central Limit Theorem. We have seen that when a sequence of chance experiments forms an independent trials process, the possible outcomes for each experiment are the same and occur with the same probability. Further, knowledge of the outcomes of the previous experiments does not influence our predictions for the outcomes of the next experiment. The distribution for the outcomes of a single experiment is sufficient to construct a tree and a tree measure for a sequence of n experiments, and we can answer any probability question about these experiments by using this tree measure. Rewrite the second paragraph in the mathematical language of probability theory.

198

Chapter 9 Dynamic Modelling In this chapter we extend economic modelling to include the time dimension. Dynamic modelling is the essence of macroeconomic theory. Our discussions provide the basic ingredients of the so-called dynamic general equilibrium model.

9.1 9.1.1

First-Order Difference Equation Constant Coefficients

Suppose that {xt } is a sequence of variables in R which satisfies the first-order difference equation xt+1 = axt + b, t = 0, 1, 2, . . . , (9.1) where a and b are constant and a 6= 0. If b = 0 the equation is called homogeneous. If the value of xt is known in period t, then all subsequent terms can be expressed in terms of xt :  n  an x t + b 1 − a if a 6= 1 1−a xt+n = (9.2)  xt + bn if a = 1 for n = 1, 2, . . . . This can be proven by mathematical induction. We first show that equation (9.2) is true for n = 1. Then by assuming that (9.2) is true for a particular period n, we show that it is also true in period n + 1. For the case of a 6= 1, it is obvious that putting n = 1 in equation (9.2) reduces 199

CHAPTER 9. DYNAMIC MODELLING it to equation (9.1). Now assume that equation (9.2) holds for xt+n , then axt+n + b = = = = = = =

  1 − an n a a xt + b +b 1−a 1 − an +b an+1 xt + ab 1 − a n  1−a n+1 a xt + b a +1 1−a   a(1 − an ) + (1 − a) n+1 a xt + b 1−a   n+1 a−a +1−a n+1 a xt + b 1−a   n+1 1−a n+1 a xt + b 1−a xt+n+1 ,

which shows that equation (9.1) holds for xt+n+1 . The proof for the case a = 1 is left as an exercise. If −1 < a < 1, then limn→∞ an = 0. By equation (9.2) we have lim xt+n =

n→∞

b . 1−a

That is, if −1 < a < 1, xt converges to a steady-state value of b/(1 − a). Figure 9.1 plots the values of xt with a = 0.3, b = 10, with initial value x0 = 1. The series quickly converges to the steady-state value of 10/(1 − 0.3) = 14.286 within ten periods. On the other hand, if a ≥ 1 or a ≤ −1, then xt diverges and does not achieve a steady state. Alternatively, equation (9.1) can be seen as a dynamical system, with xt+1 = f (xt ) = axt + b. A steady-state value for xt is a fixed point of the function f . Thus by setting xt+1 = xt = x∗ , we have b x∗ = 1−a as above. The system will converge to the fixed point x∗ if |f 0 (x∗ )| < 1 and will diverge if |f 0 (x∗ )| > 1. See Devaney (2003, p. 24) for details. 200

9.1. FIRST-ORDER DIFFERENCE EQUATION xt 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

t

Figure 9.1: First-Order Difference Equation

y

y 45◦

6

45◦

6  

r  r 

 r

 

x0 x1 x∗

 r   r  

- x

x1 x0 x∗

(a) 0 < a < 1

- x

(b) a > 1

Figure 9.2: Convergence of First-Order Difference Equation

Figure 9.2(a) illustrates the case when 0 < a < 1. The initial value of x is equal to x0 . The value of x1 = ax0 + b is reflected back to the x-axis by the 45◦ line and the process goes on. Eventually x converges to the fixed point at x∗ . Starting points with values greater than x∗ go through a similar process converging to x∗ . In figure 9.2(b) the coefficient a is greater than one. Values of x not equal to the fixed point x∗ will move away from it. You should draw the cases of −1 < a < 0 and a < −1 to reveal the dynamics of the so-called cobweb models. 201

CHAPTER 9. DYNAMIC MODELLING

9.1.2

Non-constant Coefficients

Suppose that b is not a constant but a sequence. Then (9.1) becomes xt+1 = axt + bt ,

t = 0, 1, 2, . . . ,

(9.3)

If a ≥ 1 or a ≤ −1, this difference equation can be solved “forward” to give the value of xt , that is, 1 (−bt + xt+1 ) a  1 1 = −bt + (−bt+1 + xt+2 ) a a    1 1 1 = −bt + −bt+1 + (−bt+2 + xt+3 ) a a a .. .   1 xt+s bt+1 bt+2 = − + 2 + · · · + lim s bt + s→∞ a a a a ∞ 1 X bt+s xt+s = − + lim s s s→∞ a a s=0 a

xt =

= −

∞ X bt+s xt+s + lim . s+1 s→∞ as a s=0

Convergence of this series depends on the behaviour of the sequence {bt }. In particular, a necessary condition for convergence is the so-called transversality condition, xt+s lim s = 0. s→∞ a If the transversality condition is satisfied, then xt = −

∞ X bt+s . s+1 a s=0

(9.4)

For the case of −1 < a < 1, we can solve the difference equation (9.3) “backward” to get ∞ X xt = as−1 bt−s . (9.5) s=1

In the special case that bt+s = b for all s, equation (9.3) becomes a geometric series 202

9.1. FIRST-ORDER DIFFERENCE EQUATION and so xt =

b . 1−a

See Goldberg (1986) for more on difference equations. Chapter 2 in Hamilton (1994) also contains discussions on difference equations using lag operators. Example 9.1. (Azariadis, 1993, Chapter 3) Suppose that investors have two choices of asset. The first is a riskless government bond that pays an interest at the rate of r per period. The second is a stock that pays dividend dt in period t. The stock is bought or sold at market price pt before the dividend is paid out. Then a risk-neutral investor faces (1 + r)pt = dt + Et [pt+1 ], (9.6) that is, an amount of pt invested in government bond should be equal to the expected return invested in the stock. The investor has adaptive expectation in every period: Et [pt+1 ] = λpt + (1 − λ)Et−1 [pt ],

(9.7)

with 0 < λ < 1. This means that forecasted price Et [pt+1 ] is partially adjusted by a factor of λ towards the forecasting error pt − Et−1 [pt ] in the last period. Substitute the expectation in equation (9.6) into (9.7), we get (1 + r)pt − dt = λpt + (1 − λ)[(1 + r)pt−1 − dt−1 ]. Rearranging and shifting one period forward to get pt+1 = apt + bt , where a= and bt =

(1 + r)(1 − λ) , 1+r−λ dt+1 − (1 − λ)dt . 1+r−λ

Given that r > 0, it is clear that 0 < a < 1. We can therefore apply equation (9.5) to obtain the price of the stock in period t, pt =

∞ X

as−1 bt−s .

s=1

203

CHAPTER 9. DYNAMIC MODELLING

Figure 9.3: Price/Earning Ratio of U.S. Stocks

In the special case that dividends are constant, that is, dt = d for all t, bt = b =

λd 1+r−λ

for all t. The price of the stock becomes pt =

∞ X s=1

as−1 b =

1+r−λ λd d b = = . 1−a λr 1+r−λ r

Figure 9.3 shows the price/earning ratio of the U.S. stock market, which is pt /dt in our model. The average ratio in the past 50 years is 19.5.

In passing, we should mention that higher-order difference equations can be transformed into first-order vector difference equations. For example, consider a linearly homogeneous second-order equation xt+1 = axt + bxt−1 . 204

(9.8)

9.2. DYNAMICAL SYSTEMS We can let yt = xt−1 so that the system becomes " # " #" # xt+1 a b xt = . yt+1 1 0 yt We shall discuss linear dynamical system in section 9.4.1.

9.2

Dynamical Systems

A general dynamical system can be specified as xt+1 = ft (xt , ut ),

(9.9)

yt = gt (xt , ut ),

(9.10)

where • t ∈ Z = {. . . , −2, −1, 0, 1, 2, . . .} denotes discrete time periods, • xt ∈ Rn is the state variable, • ut ∈ Rm is the input or control variable, • yt ∈ Rp is the output variable, • ft : Rn × Rm → Rn is the transition function, • gt : Rn × Rm → Rp is the payoff function, sometimes called the criterion or return function. The transition equation (9.9) relates the state variable in period t + 1 to the state and control variables in period t. The function ft is usually governed by some physical laws or technology. In an optimal control problem, the controller chooses the sequence {ut } to get a desirable sequence of outputs {yt } in equation (9.10). When there is no control variable, the system is called autonomous. In many applications, ft and gt are time-invariant or stationary and are therefore written without the time subscripts. Example 9.2. Consider a simple Ramsey model in economics. The state variable is the capital stock kt and the control variable is consumption ct . The transition equation is kt+1 = F (kt ) + (1 − δ)kt − ct , (9.11) 205

CHAPTER 9. DYNAMIC MODELLING where F is the aggregate production function and δ is the depreciation rate of capital. The output is the utility of the aggregate household so that the payoff function in each period is U (ct ). The objective of the central planner is to maximize the discounted values of present and future utility, that is, max ct

∞ X

β t U (ct ),

t=0

where β = 1/(1 + θ) is the social discount factor. The following Euler equation is obtained from the necessary conditions of optimization (see section 9.3): β

U 0 (ct+1 ) 0 [F (kt+1 ) + 1 − δ] = 1. U 0 (ct )

(9.12)

Assuming the utility function U is strictly concave so that U 00 (c) < 0 for all values of c, the Euler equation can be solved for ct+1 as a function of ct and kt (the term kt+1 can be substituted by ct and kt using equation (9.11)). The result is a twodimensional autonomous dynamical system with xt = [kt ct ]T .

Consider an autonomous and stationary dynamical system with given initial state x. After two periods we have f (f (x)) = f 2 (x). In general, f t+s (x) = f s (f t (x)). The sequence {xt } for t ∈ Z is called the orbit of x.1 When the sequence only includes t = {0, 1, 2, . . .}, it is called a forward orbit. Similarly, a backward orbit consists of t = {0, −1, −2, . . .}. A fixed point p is defined by f (p) = p. The set of all fixed points of f is written as Fix(f ). In economics a fixed point is generally called an equilibrium. In dynamic general equilibrium models, the system is sometimes linearized by first-order Taylor approximation about the steady-state values. An autonomous linear dynamical system can be written as xt+1 = Axt + But , yt = Cxt + Dut , where A, B, C and D are matrices with the conforming dimensions.2 1 2

Orbits are often called trajectories or flows. Boyd (2008) provides a detailed analysis of linear dynamical systems.

206

9.3. DYNAMIC PROGRAMMING

9.3

Dynamic Programming

In this section we assume that the payoff function g is differentiable and real-valued. That is, the output variable yt ∈ R.

9.3.1

Basic Theory

A typical dynamic programming problem is max ut

∞ X

β t g(xt , ut ),

(9.13)

t=0

subject to the transition equation xt+1 = f (xt , ut ),

(9.14)

where β ∈ (0, 1] is a discount factor. The value of the state variable in period 0, x0 , is given. In other words, we want to maximize the sum of the present values of the output variables from period t = 0 onward. Notice that the payoff function g and the transition function f are stationary. This will allow us to express the system in a recursive form below. Often we assume that g is differentiable and concave and the feasible set S = {(xt+1 , xt ) : xt+1 ≤ f (xt , ut ), t = 0, 1, . . .} is convex and compact. If the solution of the above problem exists, ut = h(xt ) is called the policy function. With the initiate state x0 and the transition equation (9.14), the sequence {(xt , ut )} is obtained. By putting the sequence {(xt , ut )} in the objective function in (9.13), we get the value function v(x0 ) = max ut

(∞ X

) β t g(xt , ut ) : xt+1 = f (xt , ut ) .

t=0

After some time periods, say s, we obtain the state variable xs . The structure of the problem remain the same since g and f are stationary. Therefore the value function, now depends on xs , is (∞ ) X v(xs ) = max β t−s g(xt , ut ) : xt+1 = f (xt , ut ) . ut

t=s

In general, in any period t, the value function is the sum of current payoff function 207

CHAPTER 9. DYNAMIC MODELLING and the next period’s value function, that is, v(xt ) = max {g(xt , ut ) + βv(xt+1 ) : xt+1 = f (xt , ut )} ut

= max {g(xt , ut ) + βv(f (xt , ut ))} . ut

(9.15)

In fact, since the value function has the same structure in every period, we can express the above equation recursively as v(x) = max {g(x, u) + βv(f (x, u))} . u

(9.16)

We can substitute the maximizer u with the policy function h(x) in the above equation to obtain v(x) = g(x, h(x)) + βv(f (x, h(x))). (9.17) Equations (9.15), (9.16), and (9.17) are different versions of the Bellman equation. Given g and f , equation (9.17) is a functional equation with unknown functions v and h to be found. Existence of the solution is confirmed by the following theorem. The proof can be found in Stokey and Lucas (1989, p. 79), and Ljungqvist and Sargent (2004, p. 1011–1012). Theorem 9.1. Suppose that g is continuous, concave, and bounded, β ∈ (0, 1), and the feasible set S is nonempty, compact, and convex. Then there exists a unique function v that solves the Bellman equation (9.17). Methods for solving the Bellman equation for the policy function h and value function v include iterations of h, iterations of v, the method of undetermined coefficients, and numerical analysis using computers. You can consult Stokey and Lucas (1989) or Ljungqvist and Sargent (2004) for details. In what follows we assume that the value function v is differentiable and derive the first-order conditions for maximizing the Bellman equation. Then we apply the envelope theorem to the value function v. Together we obtain the same set of necessary conditions as in the Lagrange method. Let λt = ∇v(xt ) and λt+1 = ∇v(xt+1 ). The first-order condition of the maximization problem in (9.15) is ∇u g(xt , ut ) + βDu f (xt , ut )T λt+1 = 0 ∈ Rm . 208

(9.18)

9.3. DYNAMIC PROGRAMMING

Table 9.1: Dimensions of Derivatives Derivatives Dimensions λt , λt+1 n×1 ∇u g(xt , ut ) m×1 ∇x g(xt , ut ) n×1 Du f (xt , ut ) n×m Dx f (xt , ut ) n×n Dh(xt ) m×n Next, we differentiate the functional equation (9.17) with respect to the state variable x: λt = ∇v(xt ) = ∇x g + (Dh)T ∇u g + β[(Dx f )T + (Dh)T (Du f )T ]λt+1 = ∇x g + β(Dx f )T λt+1 + (Dh)T [∇u g + β(Du f )T λt+1 ]. By the first-order condition (9.18), the term inside the square bracket in the last equality above is the zero vector in Rm . Therefore we have λt = ∇x g(xt , ut ) + βDx f (xt , ut )T λt+1 ∈ Rn .

(9.19)

Table 9.1 keeps track of the dimensions of the derivatives in the above algebra. You should verify that all matrix multiplications in applying the chain rule are compatible. Equation (9.19) is similar to the envelope theorem in static optimization. It effectively means that the rate of change (gradient) of the value function v with respect to the state variable xt is equal to the partial derivative of the Bellman equation with respect to xt .

9.3.2

Recipe

Here is a set of cookbook style procedures to follow as an alternative method to the Lagrange method in analyzing dynamic optimization problems: 1. Make sure that the objective function is additively separable for each period and the payoff function g(xt , ut ) is stationary. 209

CHAPTER 9. DYNAMIC MODELLING 2. Make sure that the transition function is stationary and expressed in the form xt+1 = f (xt , ut ).

(9.20)

3. Set up the Bellman equation v(xt ) = max {g(xt , ut ) + βv(f (xt , ut ))} . ut

4. Let λt = ∇v(xt ) and λt+1 = ∇v(xt+1 ). 5. Differentiate the right-hand side of the Bellman equation with respect to the control variable ut and set the gradient equal to the zero vector 0 ∈ Rm as in equation (9.18). 6. Differentiate the right-hand side of the Bellman equation with respect to the state variable xt and set the gradient equal to λt ∈ Rn as in equation (9.19). 7. The results in steps 5 and 6, together with the transition equation (9.20) form the set of necessary conditions for optimization. Example 9.3. Consider the Ramsey model in example 9.2. In this case m = n = 1. The Bellman equation is v(kt ) = max{U (ct ) + βv[F (kt ) + (1 − δ)kt − ct ]}. ct

The necessary conditions from steps 5 and 6 above are U 0 (ct ) − βλt+1 = 0, β[F 0 (kt+1 ) + 1 − δ]λt+1 = λt . From the first equation λt+1 = U 0 (ct )/β so that λt = U 0 (ct−1 )/β. Substitute these into the second equation above and shift one period forward gives the Euler equation in (9.12). The Ramsey model above can also be expressed in continuous time as Z ∞ max e−θt U (c(t)) dt, c(t)

0

210

9.3. DYNAMIC PROGRAMMING subject to dk(t) = F (k(t)) − δk(t) − c(t). dt The problem can be solved using the technique of optimal control theory and is popular among theorists of economic growth and natural resource economics. The results given by the continuous time version are similar to the discrete time version. But since economic data such as output, investment, consumption, and inflation come in discrete form, the dynamic programming approach is more convenient in empirical analysis. An excellent exposition of optimal control theory and its applications in economics can be found in Weitzman (2003).

Example 9.4. In this example we analyse the dynamics of durable goods set up in Wickens (2011, p. 74–76) using dynamic programming. Utility U (ct , Dt ) in each period depends on consumption of nondurable goods ct and the stock of durable goods Dt . The two state variables are Dt and asset holding at . The two control variables are ct and the purchase of durable goods in each period, dt , which has a relative price equal to pt . The transition equations are the durable goods accumulation and the budget constraint: Dt+1 = dt + (1 − δ)Dt , at+1 = xt + (1 + rt )at − ct − pt dt , where δ is the depreciation rate, xt is exogenous income, and rt is the interest rate. The Bellman equation is v(Dt , at ) = max {U (ct , Dt ) + βv[dt + (1 − δ)Dt , xt + (1 + rt )at − ct − pt dt ]} . ct ,dt

Let ∇v(Dt , at ) = (λD,t λa,t )T . Equation (9.18) is "

# " #" # " # Uc,t 0 −1 λD,t+1 0 +β = , 0 1 −pt λa,t+1 0

(9.21)

# " #" # " # UD,t 1−δ 0 λD,t+1 λD,t +β = , 0 0 1 + rt λa,t+1 λa,t

(9.22)

and equation (9.19) is "

211

CHAPTER 9. DYNAMIC MODELLING The square matrix in equation (9.21) can be inverted to get "

# " # 1 pt Uc,t λD,t+1 = . β Uc,t λa,t+1

Substitute this into equation (9.22), we have "

# " #" # " # 1 pt−1 Uc,t−1 UD,t 1−δ 0 pt Uc,t + = . β 0 0 1 + rt Uc,t Uc,t−1

The second line in the above equation gives the Euler equation. The first line, using the Euler equation, can be expressed as   1−δ Uc,t−1 UD,t = pt−1 − pt . β 1 + rt See Adda and Cooper (2003, Chapter 7) for a survey on the economics of durable consumption.

9.3.3

Linear Quadratic Models

A popular class of dynamic programming models is the linear quadratic model. They are useful in solving rational expectation models and as second-order approximations to nonlinear models. The structure of the model is that the payoff function g is the sum of two quadratic forms of the state variable xt and the control variables ut , while the transition function f is a linear function of xt and ut . The problem is max ut

∞ X

 T −β t xT t Rxt + ut Qut ,

β ∈ (0, 1),

t=0

subject to xt+1 = Axt + But ,

x0 given.

Here R and Q are positive semidefinite symmetric matrices of dimensions n × n and m × m respectively, A is an n × n matrix and B is an n × m matrix. The Bellman equation is   T v(xt ) = max − xT t Rxt + ut Qut + βv(Axt + But ) . ut

212

9.3. DYNAMIC PROGRAMMING Writing λt = ∇v(xt ) and λt+1 = ∇v(xt+1 ), the necessary conditions are −2Qut + βB T λt+1 = 0 ∈ Rm , T

n

−2Rxt + βA λt+1 = λt ∈ R .

(9.23) (9.24)

Premultiply equation (9.23) by B, we get βBB T λt+1 = 2BQut . Assume that m ≥ n and B has rank n. Then BB T is an n × n invertible matrix so that  −1 λt+1 = 2 βBB T BQut , and  −1 λt = 2 βBB T BQut−1 . Substitute λt and λt+1 into equation (9.24) and rearrange, we obtain  −1  −1 AT βBB T BQut − βBB T BQut−1 = Rxt ∈ Rn . This is a form of Euler equation which gives the intertemporal relation of the control variable u. If m > n, we need more constraints among the components of u such as government budget constraint, money supply rule, etc.

9.3.4

Finite Horizon Problems

In some models the optimization problem have a finite time periods. Such cases can arise in business contracts, retirement, or natural resource extraction. Suppose that the terminating time period is T , the dynamic programming problem becomes max ut

T −1 X

β t g(xt , ut ) + β T v(xT ),

t=0

subject to xt+1 = f (xt , ut ),

t = 0, . . . , T − 1.

As before the initial value of the state variable, x0 , is given. The additional term in the objective function, β T v(xT ), is the discounted scrap value, which is usually specified in the model. In fact, the problem can be solved by starting from the last 213

CHAPTER 9. DYNAMIC MODELLING period and working backward. The following example is taken from Carter (2001, Chapter 7).3 It illustrates an application of dynamic programming in operations management. The structure is similar to the “cake eating problem” described in Adda and Cooper (2003, p. 14–15). Example 9.5. A mining company has acquired a licence to extract a rare earth element for three years. The geophysicist of the company has estimated that there are 128 tonnes of the ore in the mine. The market price of the ore is $1 million per tonne. The total cost of extraction is u2t /xt , where ut is the rate of extraction and xt is the stock of the ore remaining in the ground. The discount rate β is assumed to be 1. The problem can be set up as max ut

3  X t=0

 u2t + v(x3 ) , ut − xt

subject to xt+1 = xt − ut ,

t = 0, 1, 2.

Since any quantity of ore remains in the ground after year 3 has no value to the company, we can assume that v(x3 ) = 0. The Bellman equation in period t = 2 is u2 v(x2 ) = max u − u x2 

 ,

which is maximized when u = x2 /2. Hence v(x2 ) =

x2 x2 x2 − 2 = . 2 4x2 4

The Bellman equation in period t = 1 is   u2 v(x1 ) = max u − + v(x2 ) u x1   u2 1 = max u − + (x1 − u) . u x1 4 The first-order condition is 1− 3

2u 1 − = 0, x1 4

See also Conrad (2010, p. 55).

214

9.4. AUTONOMOUS DYNAMICAL SYSTEMS which gives u = 3x1 /8 so that 3x1 9x21 1 v(x1 ) = − + 8 64x1 4



3x1 x1 − 8

 =

25 x1 . 64

Finally, the Bellman equation for t = 0 is   u2 v(x0 ) = max u − + v(x1 ) u x0   u2 25 = max u − + (x0 − u) . u x0 64 The first-order condition is 1−

2u 25 − = 0, x0 64

which gives u = 39x0 /128. The following table summarizes the stock and the optimal extraction rate in each period. t 0 1 2

5 8

xt 128 89 × 89 = 55.6

ut 39 3 × 89 = 33.4 8 5 × 89 = 27.8 16

Notice that the cost of extraction is inversely related to the stock, so it is not possible to extract all the mineral in the ground.

9.4

Autonomous Dynamical Systems

In this section we consider autonomous systems with no control variable or when the control variables follow some necessary conditions of an optimization process.

9.4.1

Linear Systems

Consider an autonomous linear dynamic system with no control variable. Assume that the linear transition function f is symmetric. Then f can be represented by a symmetric n × n square matrix A with respect to the standard basis in Rn , that is, the system is simply xt+1 = Axt . 215

CHAPTER 9. DYNAMIC MODELLING By the spectral theorem, A can be written as A = P ΛP T where Λ = diag(λ1 , λ2 , . . . , λn ), that is, Λ is a n × n diagonal matrix with the eigenvalues at the principal diagonal and the off-diagonal elements are all zero. The square matrix P = (v1 v2 · · · vn ) is an orthonormal matrix with columns formed by the corresponding normalized eigenvectors of A. The transition equation becomes xt+1 = P ΛP T xt . Geometrically, the vector xt is projected onto a different coordinate system by P T with {v1 , v2 , . . . , vn } as the basis. Then each projected coordinate of xt is multiplied by the corresponding eigenvalue λi . The rescaled coordinates are then transform back by P to the coordinates with respect to the standard basis in Rn . In period t + 2, the state variable becomes xt+2 = Axt+1 = A2 xt = (P ΛP T )(P ΛP T )xt = P Λ2 P T xt , since by orthogonality P T P = I. In general we have xt+s = P Λs P T xt . It is obvious that for the system to converge the origin, each of the eigenvalues must be less than one in absolute value. In other words, the spectral radius of A, defined as the largest of the absolute values of the eigenvalues, must be less than one. We have shown the following result.4 Theorem 9.2. Suppose that the transition function f is linear and all eigenvalues of f have absolute values less than one. Then lim f t (x) = 0

t→∞

4 Theorem 9.2 applies to asymmetric linear functions as well. See Devaney (2003) or Hasselblatt and Katok (2003, Chapter 3) for details.

216

9.4. AUTONOMOUS DYNAMICAL SYSTEMS for all x ∈ Rn . In an economic system, xt often represents the deviations a vector of variables from their steady-state values. Convergent to the origin therefore means that the system converges to a steady-state equilibrium. Example 9.6. Suppose xt+1 = Axt where 1 A= 4

! 5 3 . 3 5

The eigenvalues are λ1 = 2 and λ2 = 1/2, with corresponding eigenvectors v1 = √ √ √ √ (1/ 2, 1/ 2) and v2 = (−1/ 2, 1/ 2). In figure 9.4, S1 and S2 are the subspaces spanned by the the vectors v1 and v2 respectively. A vector xt is projected onto S1 and S2 . Its coordinate in S1 is then scaled by the the eigenvalue λ1 = 2. Similarly, the coordinate in S2 is scaled by λ2 = 1/2. In the next period xt+1 repeats the same transformation. Figure 9.4 depicts the forward orbit of such a point. The sequence is expanding by a factor of 2 in the v1 direction and contracting by a factor of 1/2 in the v2 direction. The readers are encouraged to repeat the process starting from difference points. The resulting map will become a trajectory map of the system. Notice that in the above example S1 and S2 are invariant subspaces, that is, once an orbit enter one of the subspaces, it cannot escape since Ax = λx. Formally, a set S is invariant under a transition function f if xt ∈ S implies that xt+s ∈ S for all s ≥ 1. Moreover, S1 is called an unstable subspace since any forward orbit in it diverges. On the other hand, S2 is a stable subspace since any forward orbit entering it converges to the origin.

9.4.2

Nonlinear Systems

Suppose that the transition function f : X → X, where X is a convex compact set in Rn , is differentiable. By the Brouwer fixed point theorem f has at least one fixed point. Our interest is to find out orbits in the neighbourhood of each fixed point converge to or diverge from it. It turns out the the behaviour of a nonlinear system is similar to a linear system with some generalizations. Let p be a fixed point of f , that is, f (p) = p. Let Df (p) be the derivative of f evaluated at p, which is an n × n matrix but not necessary symmetric. Then p is defined as 217

CHAPTER 9. DYNAMIC MODELLING xt+4

x2 S2

S1

xt xt+3 xt+1

xt+2

v2

v1 x1

Figure 9.4: A Linear Dynamical System 1. a sink or attracting point if all the eigenvalues of Df (p) are less than one in absolute value, 2. a source or repelling point if all the eigenvalues of Df (p) are greater than one in absolute value, 3. a saddle point if some eigenvalues of Df (p) are less than one and some are greater than one in absolute value. Theorem 9.3. Suppose f has sink at p. Then there exists an open set U with p ∈ U in which the forward orbit of any point converges to p. The largest of such open set is called the basin of attraction. The proof of theorem 9.3 for the case of R2 can be found in Devaney (2003, p. 216). A corollary of the theorem is that if p is a source and f is invertible, then there exist an open set in which the backward orbit of any point converges to p. The case for a saddle point is more interesting. Recall the example of a linear transition function in section 9.4.1 with eigenvalues λ1 = 2 and λ2 = 1/2. Therefore p = 0 is a saddle point. There we find a stable and an unstable subspaces, spanned by the eigenvectors or λ2 and λ1 respectively. The equivalent invariant subsets 218

9.4. AUTONOMOUS DYNAMICAL SYSTEMS in the nonlinear case are not linear subspaces anymore but manifolds. Manifolds are high-dimensional analogues of smooth curves (one-dimensional manifolds) or surfaces (two-dimensional manifolds). A stable manifold is often called a saddle path. Another example will illustrate the point. Example 9.7. Suppose that X ⊆ R2 and   15 3 x , 2y − x . f (x, y) = 2 8 Then f (0) = 0 so that p = 0 is a fixed point. The derivative is "

# 1/2 0 Df (x, y) = , − 45 x2 2 8 and hence

"

# 1/2 0 Df (0) = . 0 2

Thus we have a saddle point at the origin. Since f (0, t) = (0, 2t), the y-axis is the unstable manifold. Also,   t 15 3 3 3 f (t, t ) = , 2t − t 2 8  3 ! t t = , , 2 2 so that y = x3 is the stable manifold. Any orbit entering the saddle path y = x3 will converge to the origin. Otherwise it will diverge. In general, for an n-dimensional system, if Df (p) has k number of eigenvalues having absolute values less than one and n − k eigenvalues with absolute values greater than one, then there is a k-dimensional stable manifold that forward orbits converges to the fixed point p. Also, the unstable manifold will be of dimension n − k, in which backward orbits, if f is invertible, converge to p.

9.4.3

Phase Diagrams

An useful device for low-dimensional systems up to R3 is the construction of a phase diagram. It can provide useful visual insights to the overall behaviour of the system, 219

CHAPTER 9. DYNAMIC MODELLING y

y

6

∆x = 0 PP PP

6

B -

y

∆y = 0B

6 B

- x PP PP P

- x

B B B B



B B

?

(a)

(b)

- B 6 ? B 6 @ R B [email protected] [email protected] R @ PBP - x B PP P I P @ [email protected] I B @  @ B 6 ? B

(c)

Figure 9.5: Phase Diagram of a Linear System particularly for the case of a saddle point. In period t a point will move from xt to xt+1 . Therefore the vector ∆xt+1 = xt+1 − xt gives the direction and magnitude of the move from xt . Given a dynamical system f : X → X, the function d : X → X where d(x) = ∆x = f (x) − x is called a vector field. Of course p is a fixed point if and only if d(p) = 0. A phase diagram is a graph showing the vector field of a dynamical system. The first step is to identify the regions where the vector field is increasing, that is, ∆x ≥ 0. The complement of these regions are where ∆x  0. With this information the approximate locations of the stable manifold can be estimated. Example 9.8. Consider again the linear map 1 A= 4

! 5 3 . 3 5

For notational convenience let x = (x, y)T . Then ∆x = Ax − x = (A − I)x or

" # " #" # 1 1 3 x ∆x . = 4 3 1 y ∆y

Thus ∆x ≥ 0 implies that x + 3y ≥ 0 or y ≥ −x/3. Figure 9.5(a) depicts the graph of ∆x = x + 3y = 0, which is called a phase line. Points in the region above the line are increasing in the horizontal direction in their forward orbits, while points below are decreasing. On the other hand, ∆y ≥ 0 implies that 3x + y ≥ 0 or 220

9.4. AUTONOMOUS DYNAMICAL SYSTEMS y ≥ −3x. The phase line is shown in figure 9.5(b), with points above the line going upward in their forward orbits and points below going downward. Putting the two phase lines together as in figure 9.5(c), we can infer the directions of the forward orbits in different regions. The location of the saddle path can be approximated using the information, which lies in the stable subspace in the direction of v2 in figure 9.4. In a linear system with a saddle point, a phase line is a hyperplane passing through the fixed point. The hyperplane divides the whole space into two halfspaces, with orbits going in opposite directions.

Example 9.9. Consider the Ramsey model we analyzed in examples 9.2 and 9.3. We have x = [k c]T . The transition equations are the capital accumulation equation (9.11) and the Euler equation (9.12) of intertemporal consumption. In the steady state, k = k ∗ and c = c∗ . The Euler equation reduces to F 0 (k ∗ ) = θ + δ so that k ∗ = (F 0 )−1 (θ + δ). From the capital accumulation equation we have c∗ = F (k ∗ ) − δk ∗ . Using equation (9.11), ∆k ≥ 0 implies that c ≤ F (k) − δk. With the usual assumptions on the production function F (the Inada condition), the phase line ∆k = 0 has the shape shown in figure 9.6(a). Capital stock above this phase line is decreasing and below increasing. From the Euler equation, ∆c ≥ 0 ⇔ ct+1 ≥ ct ⇔ U 0 (ct+1 ) ≤ U 0 (ct ) ⇔ β(F 0 (kt+1 ) + 1 − δ) ≥ 1 ⇔ F 0 (kt+1 ) ≥ θ + δ ⇔ kt+1 ≤ (F 0 )−1 (θ + δ) ⇔ kt+1 ≤ k ∗ . 221

CHAPTER 9. DYNAMIC MODELLING c 6

c 

c

6 6

∆k = 0

∆c = 0

? k

- k

6 6   ) r ? *    6 ? k

k∗ (a)

(b)

(c)

Figure 9.6: Phase Diagram of the Ramsey Model Since kt+1 = F (kt ) + (1 − δ)kt − ct , the above result means that ∆c ≥ 0 if and only if F (k) + (1 − δ)k − c ≤ k ∗ , or c ≥ F (k) + (1 − δ)k − k ∗ . Figure 9.6(b) shows the phase line ∆c = 0. Consumption above this phase line is increasing and below decreasing. Combining all the information in figure 9.6(c), we see that the stable manifold or saddle path converges to the steady-state equilibrium (k ∗ , c∗ ) in the regions between the two phase lines. In practice, with nonlinear functional forms for U and F , an explicit formulation for the stable manifold may not be possible. Many analysts choose to linearize the system about the fixed point (k ∗ , c∗ ). Others may solve the nonlinear system by using numerical methods. The theory of dynamical systems has become an important tool in economic modelling. Dynamic general equilibrium models rely on a linearized system converging to a unique fixed point. This assumption can be too restrictive and richer and more interesting models can be achieved by considering nonlinear systems. For an introduction to the basic theory see Day (1994), Devaney (2003) and Hasselblatt and Katok (2003). For applications in economics see Azariadis (1993) and Day (1999).

9.4.4

Stochastic Processes: Once More with Feeling

In this section we introduce two applications of dynamical systems in stochastic processes. The first model, vector autoregression, is a useful tool in macroeconometrics. The second model, Markov chain, is an important ingredient in DSGE models. First 222

9.4. AUTONOMOUS DYNAMICAL SYSTEMS we define the idea of stationarity. A stochastic process is said to be covariance stationary if it satisfies two properties: 1. The mean is independent of time, E[xt ] = µ for every t. 2. The covariance matrix between two time periods, E[(xt+s − µ)(xt − µ)T ], depends on the separation of time s = 0, 1, 2, . . ., but not on t. Vector Autoregression Some economists suggest that the stock market is a leading indicator for the aggregate economy, but not the other way round.5 Moreover, equity returns are by themselves random walks. Let xt = [rt γt ]T be the rate of return of the stock market and growth rate of the economy respectively in period t and c = [c1 c2 ]T be their average rates in the long run. The relation can be represented by the model " # " # " #" # " # rt+1 c1 φ11 φ12 rt t1 = + + , γt+1 c2 φ21 φ22 γt t2 where t1 and t2 are white noises. The above ideas can be tested by the hypotheses φ11 = φ12 = 0 and φ21 > 0. In general, let xt be a state variable in period t and c be a constant vector, both in Rn . Suppose t be an n-dimensional vector of white noise, that is, 1. E[t ] = 0 ∈ Rn , n n 2. E[t T s ] = 0 ∈ R × R for t 6= s,

3. E[t T t ] = Ω, where Ω is an n × n symmetric positive definite covariance matrix. Then a firstorder vector autoregressive process, VAR(1), is given by xt = c + Φxt−1 + t ,

(9.25)

where Φ is an n × n transition matrix. It is clear that this is a multivariate version of the AR(1) process described in section 8.7.3. In view of theorem 9.2 and exercise 15 below, the process is covariance stationary if all the eigenvalues of Φ have absolute 5

See, for example, Economist (2011).

223

CHAPTER 9. DYNAMIC MODELLING values less than one. That is, xt has constant mean and variance in every period. The expected values of xt in equation (9.25) is E[xt ] = c + ΦE[xt−1 ] + E[t ], which gives µ = E[xt ] = E[xt−1 ] = (In − Φ)−1 c. Finding the covariance matrix Σ of xt is more involving but it can be shown that Σ satisfies the following discrete Lyapunov matrix equation: Σ = ΦΣΦT + Ω.

(9.26)

See Stock and Watson (2001) for an introduction in applying VAR to macroeconomics. Hamilton (1994) and L¨ utkepohl (2005) provide extensive coverage on analysis and estimation of VAR models. Markov Chains Consider a typical worker in an economy. Suppose that if the worker is employed this period, then the probability of staying employed in the next period is p11 . The probability of unemployment next period is therefore p12 = 1 − p11 . On the other hand, if the worker is unemployed in this period and the chance of finding a job in the next period is p21 , the chance of remaining unemployed is then p22 = 1 − p21 . We can represent the status xt of the worker with the standard unit vectors in R2 . That is, xt = e1 = [1 0]T means that the worker is employed in period t, and xt = e2 = [0 1]T unemployed. The conditional probabilities defined above are therefore pij = Prob(xt+1 = ej |xt = ei ),

i, j = 1, 2.

We can summarize the situation by a transition matrix " # p11 p12 P = . p21 p22

(9.27)

Now imagine we draw a random person from the work force in period t and let πt = [πt1 πt2 ]T be the probabilities that she is employed or unemployed respectively. Here πt2 = 1 − πt1 is the unemployment rate of the economy in period t. The probability distribution of this person’s employment status in the next period is 224

9.4. AUTONOMOUS DYNAMICAL SYSTEMS then

"

# " #" # π(t+1)1 p11 p21 πt1 = , π(t+1)2 p12 p22 πt2

or πt+1 = P T πt . Suppose that the average income of an employed worker is y¯1 and that of an un¯ = [¯ ¯ T xt is a random variable employed person is y¯2 . Write y y1 y¯2 ]T . Then yt = y of income in period t. To summarize, {yt } is a stochastic process of labour income with probability distribution πt in period t. In the general set up, the state of the world consists of n possible outcomes, denoted by the state vectors xt ∈ {e1 , . . . , en }. The conditional probability that state j will happen in period t + 1 given that state i occurs in period t is pij = Prob(xt+1 = ej |xt = ei ),

i, j = 1, . . . , n.

The n × n probabilities are conveniently written as the transition or stochastic matrix   p11 p12 · · · p1n  .. ..  , P =  ... (9.28) . .  pn1 pn2 · · ·

pnn

with the properties that all elements are non-negative and all the rows sum to one: n X

pij = 1,

i = 1, . . . , n.

j=1

The probability distribution of the n states in each period t is given by an n×1 vector P πt = [πt1 · · · πtn ]T , with ni=1 πti = 1. The transition equation for the probability distribution is therefore6 πt+1 = P T πt . This is an autonomous linear dynamical system on the simplex ( ) n X S = π ∈ Rn : πi ≥ 0, πi = 1 .

(9.29)

i=1 6 Some authors prefer to writing πt as a row vector so that the transition equation is πt+1 = πt P . Others define the transition matrix P as the transpose of what we specified in equation (9.28), consequently the transition equation is πt+1 = P πt .

225

CHAPTER 9. DYNAMIC MODELLING It can be shown (see exercise 23 below) that P T has at least one eigenvalue equals to one. That is, there exists an eigenvector π ∈ S such that π = P T π,

(9.30)

which implies that π is a fixed point, or in this case called a stationary probability distribution of the Markov chain. A Markov chain is called regular if some power of the transition matrix has only positive elements. That is, there is an integer k such that P k is a positive matrix. It turns out that for a regular Markov chain, all initial probability distributions converge to one unique fixed point. The process is said to be asymptotically stationary, with lim P t = Π,

t→∞

where all the n rows of Π are identical and equal to π. Now we derive the stationary probability distribution π of a regular Markov P chain. Since any probability distribution must sum to one, n1=1 πi = 1, or, if we define 1 to be an n-vector of ones, 1T π = 1.

(9.31)

(In − P T )π = 0.

(9.32)

Also, equation (9.30) implies that

Combining equations (9.31) and (9.32), we have Aπ = e1 ,

(9.33)

where e1 is the (n + 1)-dimensional standard unit vector, and "

1T A= In − P T

#

is an (n+1)×n matrix. Premultiplying both sides of equation (9.33) by (AT A)−1 AT , we get π = (AT A)−1 AT e1 . 226

9.5. EXERCISE That is, π is the first column of the matrix (AT A)−1 AT . ¯ ∈ Rn be given. Then the inner product yt = y ¯ T xt defines a real random Let y variable in period t with probability distribution πt so that {yt } is a stochastic process. Given a Markov chain with initial probability distribution π0 , the distribution after t periods is πt = (P T )t π0 . The expected value of yt is therefore ¯ = π0T P t y ¯. E[yt ] = πtT y See Grinstead and Snell (1997) and Hamilton (1994) for introductions to Markov chains, Levin, Peres, and Wilmer (2009) provides a more advanced treatment.

9.5

Exercise

1. Draw the graphs showing the dynamics of equation (9.1) for the cases of −1 < a < 0 and a < −1. 2. Prove statement (9.5). 3. Give an economic interpretation to the limiting cases λ = 0 and λ = 1 in equation (9.7). 4. Prove that Fix(f ) is a closed set if f is continuous. 5. Consider an increasing and continuous function f : X → X where X ⊆ R is a compact set. From exercise 4 above the set {x ∈ X : f (x) 6= x} is an open set and therefore a collection of open intervals. Take an interval I = (a, b) where f (a) = a, f (b) = b, and f (x) 6= x for a < x < b. Show that every forward orbit in I converges to b if f (x) > x and converges to a if f (x) < x. 6. A consumer’s savings problem is max E0 ct

∞ X

β t U (ct ),

t=0

subject to the budget constraint at+1 = (1 + rt )(at + xt − ct ), where ct is consumption, at is asset holding, xt is an exogenous income, rt is the rate of return on asset, and β = 1/(1 + θ). 227

CHAPTER 9. DYNAMIC MODELLING (a) Set up the Bellman equation and show that the Euler equation is βEt (1 + rt+1 )

U 0 (ct+1 ) = 1. U 0 (ct )

(b) Suppose that utility is represented by the quadratic function U (ct ) = −(ct − γ)2 and rt = θ for all periods. Show that consumption is a martingale, that is, Et ct+1 = ct . 7. Consider the lump-sum taxation model where the central planer maximizes household utility ∞ X  1−α β s cαt+s gt+s , 0 < α < 1, s=0

subject to the resource constraint F (kt ) = ct + kt+1 − (1 − δ)kt + gt and the government budget constraint gt = Tt . (a) Set up the Bellman equation. (b) Derive the necessary conditions from the Bellman equation. (c) Find the Euler equation, the marginal rate of substitution of household consumption to government services, and the steady-state condition. 8. Consider the following cake-eating problem: max

∞ X

ct

β t U (ct , ct−1 ),

t=1

subject to wt+1 = wt − ct with the initial size of the cake w0 given. In each period the consumer eat part of the cake (ct ) but must save the rest. (a) Set up the Bellman equation. (b) Derive the Euler equation. 9. Verify each step in example 9.4. 10. (Ljungqvist and Sargent, 2004, p. 110) Consider the linear quadratic model in section 9.3.3 with β = 1. Suppose we know that the value function is also a 228

9.5. EXERCISE quadratic form, v(x) = −xT P x, where P is a positive semidefinite symmetric matrix. (a) Set up the Bellman equation using this information. (b) Show that P = R + AT P A − AT P B Q + B T P B

−1

B T P A.

(9.34)

Equation (9.34) is call a Riccati equation in P and requires numerical analysis to solve. 11. In a linear dynamical system in R2 where |λ1 | > 1 and |λ2 | < 1, what is the necessary condition for the system to converge to the origin? 12. Solve the second-order difference equation (9.8). 13. Describe the dynamics of the following linear maps with matrix representations ! ! 1 2 1 −2 0 (a) , (b) , 1 1 0 2

(c) β

! cos α − sin α , sin α cos α

with 0 ≤ α < 2π and β > 0. 14. Suppose that all the eigenvalues a square matrix A have absolute values less than one. (a) Show that the matrix I − A is invertible. (Hint: Let (I − A)x = 0, where x is a column vector. Show that x = 0.) (b) Prove that (I − A)−1 = I + A + A2 + · · · . Note: This is a multivariate version of equation (3.3). 15. Suggest a solution for the affine dynamical system xt+1 = Axt + b, 229

CHAPTER 9. DYNAMIC MODELLING y 45◦

6

a

- x

b

Figure 9.7: The Poverty Trap where b ∈ Rn . What is the necessary condition for the system to converge to a fixed point? (Hint: This is a multivariate version of the first-order difference equation discussed in section 9.1.1.) 16. Derive the equations for the phase lines for the dynamical system in the example of section 9.4.2:   x 15 3 f (x, y) = , 2y − x . 2 8 Draw the phase diagram and locate the stable and unstable manifolds. 17. Consider the one-dimensional Solow-Swan model ∆kt+1 = sF (kt ) − (δ + n)kt , where kt is the capital stock in period t, s, δ, and n are the exogenous rates of saving, depreciation, and population growth, respectively. The production function F satisfies the Inada conditions. (a) Find the set of fixed points. (b) What are the necessary conditions for the system to converge? (c) Draw a diagram showing the dynamics of k. (d) Find the basins of attraction. 18. Growth in the Solow-Swan model is forced by the strong assumptions on the production function. Here is another model that is more realistic at low income 230

9.5. EXERCISE levels. Let xt be the income per person of an economy at period t. Suppose that xt+1 = f (xt ) where  (b − a)1−α (x − a)α + a, if a ≤ x ≤ b, f (x) = (x − b)β + b, if x > b. In the above, 0 < a < b, α > 1, and 0 < β < 1. (a) Find Fix(f ). (b) Find the basin of attraction for each fixed point. The region a ≤ x ≤ b is called a poverty trap. See figure 9.7 and Banerjee and Duflo (2011, p. 12). The book provides justifications for the shape of the growth curve. ¯ t = xt − µ, the 19. Consider a stationary VAR(1) process as in equation (9.25). Let x derivation of xt from the mean. (a) Show that the process can be represented by ¯ t = Φ¯ x xt−1 + t . (b) As in the theory of ordinary least square, the error term t is assumed to be n n ¯ t−1 , that is, E[¯ uncorrelated with x xt−1 T t ] = 0 ∈ R × R . Show that the covariance matrix Σ of xt satisfies equation (9.26). 20. Consider the following statement: The Bureau of Labour Statistics estimates that workers recently laid off have a 30% chance of finding work in a given month. For workers off the job for more than six months, that chance is no better than 10%.7 Can the employment dynamics be modelled by a Markov chain? Why or why not? If yes, what will the transition matrix look like? 21. Let (P, x) be an n-state Markov chain. Show that E[xt+1 |xt ] = P T xt . 7

“Six Years into a Lost Decade,” The Economist, August 6, 2011.

231

CHAPTER 9. DYNAMIC MODELLING 22. Show that the eigenvalues of the two-state Markov chain defined in equation (9.27) are λ1 = 1 and λ2 = p11 + p22 − 1. 23. Let P be an n × n matrix (not necessary symmetric) which has nonnegative P elements and satisfies nj=1 pij = 1 for i = 1, . . . , n. Show that P T has at least one unit eigenvalue. Hint: Consider the simplex defined in equation (9.29). Show that the linear function f represented by P T is a linear operator on S, that is, for every π ∈ S, f (π) is also in S. Then show that f : S → S satisfies the hypothesis of the Brouwer Fixed Point Theorem.

232

Bibliography Adda, J´erˆome and Russell Cooper (2003) Dynamic Economics, Cambridge: The MIT Press. Artin, Emil (1964) The Gamma Function, New York: Holt, Rinehart and Winston. Azariadis, Costas (1993) Intertemporal Macroeconomics, Cambridge: Blackwell Publishers. Banerjee, Abhijit and Esther Duflo (2011) Poor Economics: A Radical Rethinking of the Way to Fight Global Poverty, New York: PublicAffairs. Beck, Matthias and Ross Geoghegan (2010) The Art of Proof, New York: Springer Science+Business Media. Boyd, Stephen (2008) “Introduction to Linear Dynamical Systems,” Lecture Notes for EE263, Stanford University. (Lectures available on iTune U) Box, George E.P., Gwilym M. Jenkins, and Gregory C. Reinsel (1994) Time Series Analysis: Forecasting and Control, Third edition, Englewood Cliffs: PrenticeHall, Inc. Brosowski, Bruno and Frank Deutsch (1981) “An Elementary Proof of the StoneWeierstrass Theorem,” Proceedings of the American Mathematical Society, 81(1), 89–92. Carter, Michael (2001) Foundations of Mathematical Economics, Cambridge: The MIT Press. Clausen, Andrew (2006) “Log-Linear Approximations,” Unpublished class note, University of Pennsylvania. Conrad, Jon M. (2010) Resource Economics, Second edition, Cambridge: Cambridge University Press. Day, Richard H. (1994) Complex Economic Dynamics, Volume I, Cambridge: The MIT Press. 233

Bibliography Day, Richard H. (1999) Complex Economic Dynamics, Volume II, Cambridge: The MIT Press. Debreu, Gerard (1952) “Definite and Semidefinite Quadratic Forms,” Econometrica, (20)2, 295–300. Devaney, Robert L. (2003) An Introduction to Chaotic Dynamical Systems, Second Edition, Cambridge: Westview Press. Devlin, Keith (1993) The Joy of Sets, Second Edition, New York: Springer Science+Business Media. Devlin, Keith (2002) “Kurt G¨odel—Separating Truth from Proof in Mathematics,” Science, Volume 298, December 6 issue, 1899–1900. Diewert, W.E. (1999) “Unconstrained Optimization,” Unpublished Lecture Notes, University of British Columbia. Economist (1999) “Getting the Goat,” February 18 issue. Economist (2008) “Easy as 1, 2, 3,” December 30 issue. Economist (2011) “The Missing Link,” May 19 issue. Ellis, George (2012) “On the Philosophy of Cosmology,” Talk at Granada Meeting, 2011. Available at Gerstein, Larry J. (2012) Introduction to Mathematical Structures and Proofs, Second Edition, New York: Springer Science+Business Media. Goldberg, Samuel (1986) Introduction to Difference Equations, New York: Dover Publications. Grinstead, Charles M. and J. Laurie Snell (1997) Introduction to Probability, Second Revised Edition, Providence: American Mathematical Society. (Downloadable from the Chance web site.) Hamilton, James D. (1994) Time Series Analysis, Princeton: Princeton University Press. Hasselblatt, Boris and Anatole Katok (2003) A First Course in Dynamcis: with a Panorama of Recent Developments, Cambridge: Cambridge University Press. Hoffman, Kenneth and Ray Kunze (1971) Linear Algebra, Second Edition, Englewood Cliffs: Prentice-Hall. 234

Bibliography Hogg, Robert V. and Allen T. Craig (1995) Introduction to Mathematical Statistics, Fifth Edition, Englewood: Prentice-Hall, Inc. Holt, Jim (2008) “Numbers Guy,” The New Yorker, March 3 issue. Jehle, Geoffrey A. and Philip J. Reny (2011) Advanced Microeconomic Theory, Third edition, Harlow: Pearson Education Limited. Johnston, J. (1991) Econometric Methods, Third Edition, New York: McGraw-Hill. Lancaster, Kelvin (1968) Mathematical Economics, New York: The Macmillan Company. Lay, Steven R. (2000) Analysis with an Introduction to Proof, Third edition, Upper Saddle River: Prentice Hall. Levin, David A., Yuval Peres, and Elizabeth L. Wilmer (2009) Markov Chains and Mixing Times, Providence: American Mathematical Society. (Chapters 1 and 2 downloadable from the AMS web site.) Ljungqvist, Lars and Thomas J. Sargent (2004) Recursive Macroeconomic Theory, Second Edition, Cambridge: The MIT Press. L¨ utkepohl, Helmut (2005) New Introduction to Multiple Time Series Analysis, Berlin: Springer-Verlag. Marsden, Jerrold E. and Anthony J. Tromba (1988) Vector Calculus, Third edition, W.H. Freeman and Company. Matso, John (2007) “Strange but True: Infinity Comes in Different Sizes,” Scientific American, July 19 issue. McLennan, Andrew (2014) Advanced Fixed Point Theory for Economics, Available at . Overbye, Dennis (2014) “In the End, It All Adds Up to −1/12,” The New York Times, February 3 issue. Paulos, John Allen (2011) “The Mathematics of Changing Your Mind,” The New York Times, August 5 issue. Poston, Tim and Ian Stewart (1978) Catastrophe Theory and its Applications, London: Pitman. Rao, C.R. (1973) Linear Statistical Inference and Its Applications, Second Edition, New York: John Wiley & Sons. 235

Bibliography Rockafellar, R. Tyrrell (1970) Convex Analysis, Princeton: Princeton University Press. Rosenlicht, Maxwell (1968) Introduction to Analysis, Scott, Foresman and Co. (1986 Dover edition). Ross, John F. (2004) “Pascal’s legacy,” EMBO Reports, Vol. 5, Special issue, S7– S10. Royden, H.L. and P.M. Fitzpatrick (2010) Real Analysis, Boston: Prentice Hall. Rudin, Walter (1976) Principles of Mathematical Analysis, Third edition, New York: McGraw-Hill. Starr, Ross M. (2008) “Shapley-Folkman Theorem,” in The New Palgrave Dictionary of Economics, Second Edition, Houndmills: Palgrave Macmillan. Stock, James H. and Mark W. Watson (2001) “Vector Autoregressions,” Journal of Economic Perspectives, 15(4), 101–115. Stokey, Nancy L. and Robert E. Lucas (1989) Recursive Methods in Economic Dynamics, Cambridge: Harvard University Press. Strogatz, Steven (2010) “Division and Its Discontents,” The New York Times, February 21 issue. Sundaram, Rangarajan K. (1996) A First Course in Optimization Theory, Cambridge: Cambridge University Press. Taylor, Howard M. and Samuel Karlin (1998) An Introduction to Stochastic Modeling, Third Edition, San Diego: Academic Press. Timoshenko, S. (1940) Strength of Materials, New York: D. Van Nostrand Company. Tversky, Amos and Daniel Kahneman (1974) “Judgment under Uncertainty: Heuristics and Biases,” Science, 185(4157), 1124–1131. Weitzman, Martin L. (2003) Income, Wealth, and the Maximum Principle, Cambridge: Harvard University Press. Wickens, Michael (2011) Macroeconomic Theory: A Dynamic General Equilibrium Approach, Second Edition, Princeton: Princeton University Press. Wiggins, Chris (2006) “Bayes’s theorem,” Scientific American, December 4 issue.

236