GPT Retakes My Midterm and Gets an A

When the answers change, I change my mind.

ChatGPT scored poorly on my Fall, 2022 Labor Economics midterm. A D, to be precise. The performance was so poor compared to the hype that I publicly bet Matthew Barnett that no AI would be able to get A’s on 5 out of 6 of my exams by January of 2029. Three months have passed since then. GPT-4 has been released. Bet On It reader Collin Gray has kindly used GPT-4 to re-run the same test.

To my surprise and no small dismay, GPT-4 got an A. It earned 73/100, which would have been the fourth-highest score on the test. The most natural explanation to my mind was that my blog post made it into the new training data, but multiple knowledgeable friends assure me that there is no new training data.

I wouldn’t have been surprised by a C this year, a B in three years, and a 50/50 A/B mix by 2029. An A already? Base rates have clearly failed me. I’m not conceding the bet, because I still think there’s a 10-15% chance I win via luck. (And in any case, high inflation keeps eroding the real value of repayment).

But make no mistake, this software truly is the exception that proves the rule. (It also sharply raises my odds that the next version of DALL-E will be able to illustrate my third graphic novel, which would be a great blessing).

Here are the results, with commentary.

---

Economics 321 Midterm

Prof. Bryan Caplan

Fall 2022

Score             Approximate Grade

60-100            A

45-59              B

35-44              C

25-34              D

0-24                F

GPT-4’s Total Score: 73/100

Part 1: True, False, and Explain

(10 points each - 2 for the right answer, and 8 for the explanation)

State whether each of the following six propositions is true or false.  In 2-3 sentences, explain why.  Use diagrams if helpful.

1. Suppose a country’s workplaces start to experience periodic blackouts.

T, F, and Explain: Aggregate Labor Demand will clearly fall even though some workers will benefit.

Suggested Answer:

TRUE.  ALD will definitely fall because MPP is the sole cause of MVP at the aggregate level, and workers are less productive with intermittent electricity.  However, workers in industries with inelastic product demand will benefit because MVP=MPP*P, and P will rise more than MPP falls.  Electrical repairmen are an obvious example.

GPT-4 Answer:

True. Aggregate Labor Demand will likely fall in response to periodic blackouts. This is because blackouts disrupt the normal functioning of workplaces, leading to lower productivity and efficiency. While some workers may experience temporary benefits such as reduced working hours, the overall negative impact on the economy and labor demand will likely outweigh these individual gains.

Score: 6/10. GPT-4 got the main idea right, but didn’t identity workers in industries with inelastic product demand as the key beneficiaries.

2.  “Productivity growth in one sector can very easily reduce employment in that sector.” (Krugman, The Accidental Theorist)

T, F, and Explain:  Krugman argues that such employment loss is a market failure that justifies government regulation.

Suggested Answer:

FALSE.  According to Krugman, employment will fall in some sectors, but rise in others.  The net result of higher productivity, as usual, is that living standards rise overall, though not everyone gains.  See his hotdog/bun thought experiment.

GPT-4 Answer:

False: Krugman does not argue that employment loss due to productivity growth is a market failure that justifies government regulation. Instead, he suggests that this phenomenon is a natural outcome of technological progress and market dynamics. The statement highlights the fact that productivity growth can lead to labor-saving innovations, which may reduce the need for human labor in a given sector. However, Krugman does not imply that this requires government intervention.

Score: 10/10. That’s exactly what Krugman said.

3.  Suppose the government starts licensing car mechanics, but this fails to raise the quality of their work.

T, F, and Explain:  Mechanics’ wages will definitely rise.

Suggested Answer:

TRUE.  Licensing reduces supply, which raises wages and reduces quantity.  If licensing raised quality, demand would go up, too.  But by assumption, that didn’t happen.  (And if it did, it would only reinforce the conclusion that wages go up!)

GPT-4 Answer:

False: If the government starts licensing car mechanics but this fails to raise the quality of their work, it is not guaranteed that mechanics' wages will definitely rise. While licensing can restrict the supply of mechanics by creating entry barriers, wages will not necessarily increase if the quality of work remains unchanged. Consumers may be unwilling to pay higher prices for the same quality of service, and the demand for mechanics may remain stable or even decrease if the public perceives the licensing as ineffective.

Score: 3/10. The AI fails to apply basic supply-and-demand. Since quality at least does not fall, falling supply and non-falling demand ensures higher wages for mechanics.

4. In his discussion of immigration, Landsburg remarks: "[W]e ostentatiously ignore the most obvious beneficiaries of open borders and can still conclude that open borders are a good thing.”

T, F, and Explain:  According to Landsburg, the "most obvious beneficiaries" are highly skilled American workers.

Suggested Answer:

FALSE.  Landsburg says that the “most obvious beneficiaries” are the immigrants themselves, who get a massive increase in pay.  His point is that the net benefits for everyone else are ALSO positive, so immigration looks like a big social gain even if we ignore the most obvious gainers.

GPT-4 Answer:

False: According to Landsburg, the "most obvious beneficiaries" of open borders are not highly skilled American workers but rather the immigrants themselves. They are the ones who benefit from better economic opportunities, higher wages, and improved living conditions in the destination country. Landsburg's statement suggests that even without considering the direct benefits to immigrants, open borders can still be seen as advantageous due to the positive effects on the overall economy, including increased diversity, a larger labor pool, and potential gains in productivity.

Score: 10/10. That’s exactly what Landsburg said.

5. Suppose you’re a typical selfish person.

T, F, and Explain:  Evidence of severe credit market imperfections will make you more eager to continue your education, but evidence of severe externalities of education will not.

Suggested Answer:

TRUE.  If there are severe credit market imperfections, the private return to education will consistently exceed the ordinary investment return throughout the rest of the economy.  The idea is that due to lack of collateral, students might be unable to get the loans they need to capture this return, which keeps returns elevated.  Knowledge of this elevated return is an extra reason to stay in school.  Externalities, in contrast, by definition, don’t affect private returns, so selfish students won’t care about them.

GPT-4 Answer:

False: If you are a typical selfish person, evidence of severe credit market imperfections might make you less eager to continue your education. This is because credit market imperfections can make it difficult for you to obtain loans or financing for your education, which could increase the cost or risk associated with pursuing higher education. On the other hand, evidence of severe externalities of education may not directly affect your decision since externalities refer to the effects of your education on others (positive or negative), which a selfish person might not consider in their decision-making process.

Score: 4/10. The AI misses the key point that credit market imperfections imply that the return to education exceeds the normal market rate, amplifying the individual motive to acquire degrees. The background assumption is that the individual knows how hard it is to pursue education personally; discovering that severe credit market imperfections exist system-wide implies greater payoffs of success. The AI does, however, get the externalities part right.

6. Higher-IQ students have a higher college completion probability.

T, F, and Explain:  This does not affect college students’ expected rate of return.

Suggested Answer:

FALSE.  The rate of return to education depends on both (a) the gains of school if you graduate, and (b) the probability of graduating.  If high IQ raises (b), then its expected return automatically rises as well.  It’s like investing in a restaurant: Profitability depends on (a) the profits if the restaurant survives, and (b) the probability that the restaurant survives.

GPT-4 Answer:

False: The fact that higher-IQ students have a higher college completion probability does affect college students' expected rate of return. Higher-IQ students are more likely to complete their degrees and therefore more likely to benefit from the increased earning potential associated with having a college degree. As a result, the expected rate of return for higher-IQ students will be higher compared to students with lower completion probabilities, given the greater likelihood of reaping the benefits of their investment in education.

Score: 8/10. Basically right, but I took off 2 points for failing to explain the logic of expected returns.

Part 2: Short Answer

(20 points each)

In 4-6 sentences, answer both of the following questions.  Use diagrams if helpful.

1. The U.S. has had two years of very high inflation.  Use a clearly-labelled supply-and-demand diagram to show what has been happening to the real minimum wage from 2019-2022.  What will ultimately happen to low-skilled workers’ living standards and employment if high inflation continues and the federal government fails to raise the minimum wage?  Consider all of the notable effects.

Suggested Answer:

During this time, the real minimum wage has been rapidly falling.  As a result, the disemployment effect of the minimum wage has been falling as well.  Real wages are getting closer to market-clearing levels, and employment is getting closer to full employment.  If inflation continues and the government doesn’t raise the minimum wage, this will ultimately amount to the abolition of the minimum wage.  The result: Wages will fall to the market-clearing level, and employment will reach full employment.  The effective abolition of the minimum wage is bad for lucky low-skilled workers who currently get higher pay, but better for everyone else, including consumers, unlucky low-skilled workers who can’t find work at all, employers, investors, and anyone who wants basic job training.  (See graph on the next page).

GPT-4 Answer:

Since I cannot create diagrams, I will describe the situation using words. From 2019 to 2022, the US experienced high inflation. During this period, if the nominal minimum wage remained constant, the real minimum wage (adjusted for inflation) would have fallen. This is because the purchasing power of a fixed nominal wage decreases as the general price level rises.

A clearly-labelled supply-and-demand diagram would show the labor market with the wage on the vertical axis and the quantity of labor on the horizontal axis. The supply curve slopes upward, representing the positive relationship between wages and the quantity of labor supplied. The demand curve slopes downward, indicating the negative relationship between wages and the quantity of labor demanded. The real minimum wage would be a horizontal line, initially set at a level above the equilibrium wage for low-skilled workers.

If high inflation continues and the federal government fails to raise the minimum wage, the real value of the minimum wage will continue to decline. This may result in low-skilled workers' living standards deteriorating, as their wages lose purchasing power. However, as the real minimum wage decreases, the demand for low-skilled labor might increase, potentially leading to higher employment levels for these workers. The overall effect on low-skilled workers will depend on the balance between the loss in purchasing power and the potential increase in employment opportunities.

Score: 17/20. Almost perfect. The AI lost points for mixing up changes in demand with changes in the quantity demanded. I almost took off 2 more points for failing to mention effects on employers and investors, but then I realized that the question never requested those.

2. Suppose there are two goods – meals and steel. Here is how much American and Mexican workers can produce in an hour:

                  Steel    Meals 

American   7          3 

Mexican     1          2 

Give a simple example showing how Mexican immigration effectively increases American workers’ productivity.  Why is this important?  (Hint: How important are services in modern economies?)

Suggested Answer:

Americans have a comparative advantage in Steel, Mexicans in Meals, and a price ratio of 1:1 would make it profitable for each country to specialize in its comparative advantage.  As a result, if an American worker switches from spending one hour each on Steel and Meals to producing Steel exclusively, he goes from producing 7 Steel and 3 Meals to producing 14 Steel.  This allows him to purchase 3 Meals and still be 4 Steel richer than he started.

This is important because in modern economies, services are 80% of employment.  So if you can’t trade services, most potential gains to trade go unrealized.  Free trade in goods without free trade in labor is a pale imitation of actual free trade.

GPT-4 Answer:

In this example, we can see that American workers are more productive in producing steel, while Mexican workers are more productive in producing meals. The productivity of each group of workers can be measured by their comparative advantage in producing each good.

Let's assume one American worker and one Mexican worker work together for an hour. If the American worker focuses on producing steel, they can produce 7 units, while the Mexican worker focuses on producing meals and can produce 2 units. In this scenario, the total output is 7 units of steel and 2 meals. If instead, the American worker spent half of their time on each good, they would produce 3.5 units of steel and 1.5 meals. The Mexican worker would produce 0.5 units of steel and 1 meal in the same time. The combined output in this case would be only 4 units of steel and 2.5 meals.

By allowing Mexican immigration, the American workers can specialize in producing steel, where they have a comparative advantage, while Mexican workers can specialize in producing meals, where they have a comparative advantage. This specialization effectively increases the productivity of both American and Mexican workers.

This is important because services, such as meal preparation, are a significant part of modern economies. As the economy grows and evolves, the allocation of labor resources becomes increasingly vital. By allowing workers to focus on their areas of comparative advantage, the overall productivity of the economy increases, leading to higher living standards and greater economic growth.

Score: 15/20. The AI lost five points for failing to explain that (a) services are a large majority of jobs in modern economies, and (b) without immigration, most services can’t be internationally traded.