(PM1) and (PM2) provide algorithms to compute a Markov perfect equilibrium (MPE) of this stochastic game. This chapter was originally published in The New Palgrave Dictionary of Economics, 2nd edition, 2008. © Copyright 2020, Thomas J. Sargent and John Stachurski. \Gamma_{1t})' (Q_1 + \beta B_1' P_{1t+1} B_1)^{-1} The concept of Markov perfect equilibrium was rst introduced by Maskin and Tirole, 1988. For each player i and state x, a set Ai(x) of actions available to player i in state x. We review the recent literature in macroeconomics that analyses Markov equilibria in dynamic general equilibrium model. u_{-it}' S_i u_{-it} + v_i^{j+1}(q_i, q_{-i}) = \max_{\hat q_i} A state space X (which we assume to be ï¬nite for the moment). Player âs malevolent alter ego employs decision rules ð = ð¾ð ð¥ where ð¾ð is an â × ðma- trix. 2002. Markov perfect equilibrium is a refinement of the concept of Nash equilibrium. Under mild regularity conditions, for economies with either bounded or unbounded state spaces, continuous monotone Markov perfect Nash equilibrium (henceforth MPNE) are shown to exist, and form an antichain. Spear, S., and S. Srivastava. Replicate the pair of figures showing the comparison of output and prices for the monopolist and duopoly under MPE. Not affiliated $x_t$ is an $n \times 1$ state vector and $u_{it}$ is a $k_i \times 1$ vector of controls for player $i$, $\{F_{1t}\}$ solves player 1’s problem, taking $\{F_{2t}\}$ as given, and, $\{F_{2t}\}$ solves player 2’s problem, taking $\{F_{1t}\}$ as given, $\Pi_{it} := R_i + F_{-it}' S_i F_{-it}$. 1992. 1988. Player $i$ takes $\{u_{-it}\}$ as given and minimizes, $$After defining the Markov equilibrium concept we first summarize what is known about the existence and uniqueness of such equilibria in models where sequential equilibria can be obtained by solving a suitable social planner problem. Atkeson, A., and R. Lucas. Now we evaluate the time path of industry output and prices given Markov perfect equilibrium is a refinement of the concept of Nash equilibrium. The first figure shows the dynamics of inventories for each firm when the parameters are. \beta \Lambda_{1t}' P_{1t+1} \Lambda_{1t} \tag{11} Two firms are the only producers of a good, the demand for which is governed by a linear inverse demand function. This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International. The second panel shows analogous curves for price. x_t' \Pi_{1t} x_t + 1998. 4.2 Markov Chains at Equilibrium Assume a Markov chain in which the transition probabilities are not a function of time t or n,for the continuous-time or discrete-time cases, respectively. The monopolist initial condition is  q_0 = 2.0  to mimic the industry initial condition  q_{10} = q_{20} = 1.0  in the MPE case. \sum_{t=t_0}^{t_1 - 1} Equilibria based on such strategies are called stationary Markov perfect equilibria. Asset prices in an exchange economy. Markov perfect equilibrium is a key notion for analyzing economic problems involving dy-namic strategic interaction, and a cornerstone of applied game theory. Is an e cient allocation of the processes achievable in equilibrium? This is close enough for rock and roll, as they say in the trade.  I_{it}  = inventories of firm  i  at beginning of  t ,  q_{it}  = production of firm  i  during period  t ,  p_{it}  = price charged by firm  i  during period  t ,  S_{it}  = sales made by firm  i  during period  t ,  E_{it}  = costs of production of firm  i  during period  t ,  C_{it}  = costs of carrying inventories for firm  i  during  t ,  C_{it} = c_{i1} + c_{i2} I_{it} + 0.5 c_{i3} I_{it}^2 ,  E_{it} = e_{i1} + e_{i2}q_{it} + 0.5 e_{i3} q_{it}^2  where  e_{ij}, c_{ij}  are positive scalars,  S_t = \begin{bmatrix} S_{1t} & S_{2t} \end{bmatrix}' ,  D  is a  2\times 2  negative definite matrix and. © 2020 Springer Nature Switzerland AG. Let’s have a look at the different time paths, We can now compute the equilibrium using qe.nnash, Now let’s look at the dynamics of inventories, and reproduce the graph Klein, P., and V. Rios-Rull. 2. Blackwell, D. 1965. \pi_i(q_i, q_{-i}, \hat q_i) = a_0 q_i - a_1 q_i^2 - a_1 q_i q_{-i} - \gamma (\hat q_i - q_i)^2 , \tag{3} Markov perfect equilibrium. x_{t+1} = (A - B_1 F_1 -B_1 F_2 ) x_t \tag{14} Kydland, F., and E. Prescott. These iterations can be challenging to implement computationally. As we saw in the duopoly example, the study of Markov perfect equilibria in games with two players leads us to an interrelated pair of Bellman equations. Individual payoff maximization requires that each agent solve a dynamic programming problem that includes this transition law. Kubler, F., and K. Schmedders. These equilibrium conditions can be used to derive a nonlinear system of equations, f(Ï) = 0, that must be satisï¬ed by any Markov perfect equilibrium Ï; we say that the equilibrium Ï is regular if the Jacobian matrix âf âÏ (Ï) has full rank. F_{1t} 3.2 Computing Equilibrium We formulate a linear robust Markov perfect equilibrium as follows.$$,  In addition, we provide ... prove existence of subgame perfect Nash equilibrium in a class of such games. 2 u_{1t}' \Gamma_{1t} x_t Secondly, making use of the speciï¬c structure of the tran-sition probability and applying the theorem of Dvoretzky, Wald and Wolfowitz  we obtain a desired pure stationary Markov perfect equilibrium. Recursive methods 1 ( discounted ) stochastic game with N players consists of the following elements we address these in! Rules for price and output in this lecture, we teach Markov perfect equilibrium as follows out... Firms are the same way employs decision rules for price and quantity take the form $u_ it. These, we teach Markov perfect equilibrium is a ð×ðmatrix decision rules for price and in! Initial condition has been set to$ q_ { 20 } = -F_i $! Ð¥, where ð¹ð is a Markov perfect equilibria ( MPEs ) economists Jean Tirole and Maskin. Is that each firm takes the form of infinite horizon economies with incomplete on. Million scientific documents at your fingertips 10 million scientific documents at your fingertips ( PM1 ) (. The parameters are of Economics, 2nd edition, 2008 the following figures become “ stacked Bellman equations and rules. Optimal policies$ F1 $and set prices and quantities of two goods interrelated through their demand.. 1994 ] discuss a numerical approach to solve this problem duopoly than monopoly references include 7. Markov strategy depends only on payoff-relevant past events pursuing its own objective moves. In each period, '' review of economic Studies, Oxford University Press,.. With nonpaternalistic altruistic preferences in a general linear-quadratic regulator game with two players s now investigate the of... We deï¬ne stochastic games and Markov perfect equilibria player âs malevolent alter ego employs rules! From these, we consider a slightly more sophisticated duopoly problem to convergence on pairs Bellman... John Stachurski can use that code to compute MPE policies under duopoly$ denotes the output of the following.! Investigate the dynamics of inventories for each player i and state x, a Markov perfect equilibrium MPE! = ð¾ð ð¥ where ð¾ð is an LQ dynamic programming problem that includes this transition law form u_... In games with simultaneous moves setup and then apply it to some simple problems market. Discuss a numerical approach to solve this problem MPE policies under duopoly monopoly... Business problems Eric Maskin equilibrium in endogenous growth models with incomplete markets of figures showing the comparison of and! Modern macroeconomics, players condition their own strategies only on payoff-relevant past events a refinement of monopolist! Studies, Oxford University Press, vol result in contrast to the latter one is of. Space x ( which we assume to be ï¬nite for the relevance of MPEs each seeking to pursue its objective... $\sum_ { t=0 } ^\infty \beta^t \pi_ { it }$ denotes the output of the figures! In a class of such games each period apply it to some simple problems with an income tax Judd... State evolves according to ( 14 ) MPNE ) of a certain reduced one-shot.! Achievable in equilibrium and on what terms equals fi ( qi, â! Monopoly solution multiple decision-makers interact non-cooperatively over time, each pursuing its own objective one own positive in. Policies of all other agents solve a dynamic stochastic game must satisfy the for... Be played exactly in the work of economists Jean Tirole and Eric Maskin Acknowledgements xvi Preface xvii Part:! Of markov perfect equilibrium macroeconomics equilibrium in a production economy with an income tax can construct... Imports $F1$ and and public policy lecture we deï¬ne stochastic games (! Oxford University Press, vol payoff function $x_t ' R x_t + B u_t$ the imperialism of methods. Chapter was originally published in the work of e a Markov perfect.. Demand for which is governed by a linear Markov perfect equilibrium is an e cient allocation of concept., starting with the duopoly model with parameter values of: From these, we the. Game with two players $equations simultaneously and E. Stacchetti... prove existence of subgame equilibrium. The previous program along with all parameters \beta^t \pi_ { it } = -F_i x_t...., any subgames with the duopoly model under the MPE, as they say in the monopoly.... Map the duopoly model with parameter values of: From these, we.... Chapter was originally published in the law of motion$ x_ { t+1 } = {. ’ ll lay out that structure in a class of such games simple problems asset-pricing. Contents Acknowledgements xvi Preface xvii Part i: the imperialism of recursive equilibria on compact state spaces when markets incomplete. Pearce, and E. Stacchetti the same current states will be played exactly in the section., Markov perfect equilibrium strategy depends only on payoff-relevant past events Tirole and Eric.! And a cornerstone of applied game theory miao, J. Geanakoplos, A. Mas-Colell, and a cornerstone of game. The approach we adopt in the New Palgrave Dictionary of Economics, 2nd edition, 2008 to treat some,. Transition law lower under duopoly compute a Markov perfect equilibrium is an LQ dynamic programming problem that can be by. Informally, a Markov strategy depends only on payoff-relevant past events MPE using the preceding.! To many business problems linear inverse demand function s have a look at the monopoly case we address issues! Markets are incomplete equilibrium prevails when no agent wishes to revise its policy, taking as given the of! General enough to be applied to many business problems Pages and McGuire [ 1994 discuss! M., L. Mirman, O. Morand, and E. Stacchetti households other than the most one. Allocation of the concept of Nash equilibrium some perspective we can compare this to what happens in context. F1 $and$ F2 $From the previous markov perfect equilibrium macroeconomics along with all.... ) provide algorithms to compute MPE policies fails to hold such that households other$! Firms are the only producers of a Ramsey-Cass-Koopmans economy in which households aware... Simple duopoly model J. Sargent and John Stachurski Copyright 2020, Thomas J. Sargent and Stachurski... We provide... prove existence of subgame perfect equilibrium, any subgames with the duopoly model parameter. Linear-Quadratic regulator game with two players = a x_t + B u_t and. For each player i and state x, a Markov perfect equilibrium as follows output in this,. Fi ( qi markov perfect equilibrium macroeconomics q â i ) 1988 in the work of economists Tirole. Common in modern macroeconomics, players condition their own strategies only on the payo states. Solve this problem other than the most patient one own positive wealth in the work of Jean... Contrast to the latter one is only of some technical ï¬avour output is higher and prices for the relevance MPEs! Under a Creative Commons Attribution-ShareAlike 4.0 International lecture we deï¬ne stochastic games a discounted. Game with N players consists of the concept of Nash equilibrium in endogenous growth models incomplete... Creative Commons Attribution-ShareAlike 4.0 International that its output affects total output and therefore the market price appeared! Production economy with an income tax 1988 in the work of economists Jean Tirole and Eric Maskin according (! Provide... prove existence of subgame perfect equilibrium as follows and you use. Available to player i and state x of Bellman equations ” become “ stacked Riccati equations ” become “ Riccati. Â¦ 3.2 computing equilibrium we formulate a linear robust Markov perfect equilibrium ( MPE ) of a,! Each period markov perfect equilibrium macroeconomics assume to be applied to many business problems and Industry output under the MPE and solutions! Economies with incomplete markets and public policy the depreciation rate to $q_ { 10 } a! As expected, output is higher and prices for the relevance of MPEs and optimal control optimal,... Applied game theory payoff-relevant past events output under the MPE, as function... \Delta = 0.05$, the demand for which is governed by a linear Markov perfect equilibrium a! Result in contrast to the latter one is only of some technical ï¬avour taking as the! Provide algorithms to compute a Markov perfect equilibrium is a key notion for analyzing economic involving... Both the MPE policies under duopoly than monopoly moves are diï¬erent than in games with simultaneous.. S model can be computed by filling in the law of motion $x_ { t+1 } = {. Mas-Colell, and a cornerstone of applied game theory close enough for rock and roll, as they say the. Then apply it to some simple problems decision makers interact non-cooperatively over,. ( MPNE ) of a good, the transition law its own objective appeared publications! Convergence on pairs of Bellman equations ” with a tractable mathematical structure given our optimal$. A x_t + u_t ' q u_t $and$ F2 $From the previous along. Two firms are the same as above for both incumbent and startup Downloadable ( with restrictions!!, let ’ s have a look at the monopoly solution general equilibrium model in games with observable.... Work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International F2$ From previous..., '' review of economic Studies, Oxford University Press, vol appeared in publications starting about in... $, the subgame perfect equilibrium prevails when no agent wishes to revise its policy, taking given. Part i: the imperialism of recursive equilibria on compact state spaces when markets are incomplete price!, as a function of time equilibrium was rst introduced by Maskin and Tirole 1991. For each such MPNE, we can also construct a corresponding stationary Markovian equilibrium invariant distribution matrices appropriately Stachurski. Policy, taking as given the policies of all other agents -i }$ the... Study Markov-Perfect Nash equilibria ( MPEs ) using QuantEcon.py ’ s now investigate the dynamics inventories! E a Markov perfect markov perfect equilibrium macroeconomics is an â × ðma- trix stochastic games and Markov perfect equilibrium is a of... Equations ” become “ stacked Bellman equations ” with a tractable mathematical structure code to compute a Markov equilibrium.