The House: Too Few Hot Races or Tidal Wave? Hard Numbers

There are two views floating around on whether we can take the House.  The first view, offered most notably by Charlie Cook (see http://www.cookpolitical.com/...) is that there just aren't enough competitive races; realistically, the Dems can pick up 4 to 9 seats.  The counterargument says this misses the forest for the trees.  If this is a "tidal wave" election analogous to 1994 (similar dynamics if not the same magnitude), races that don't look competitive now will be, and races that look competitive could go blue en masse.  

Now, I think the Tidal Wavers are clearly right in some sense.  If we outpoll the Rs in House races in aggregate by 15% (virtually impossible), we'll get a majority of the House.  So, really the question is this: what margin in the aggregate voting would be enough to let the Ds take the house?  15% would do it, but that's not realistic.  4% is realistic, but is it enough?  If we could answer this question, then maybe we could use generic congressional polls (GCPs) to start handicapping the Ds chances of a take over.  This would provide an alternative analytical framework to the Cook-style race-by-race approach.  

Well, I've crunched a lot of numbers.  And, I think I've got a good handle on what kind of aggregate margin we need to take the House.  (For those who can't wait, its around 5%.)  I started by studying aggregate vote margins over the past sixty years and how they correlate to the allocation of seats.  (The raw data is available at http://clerk.house.gov/....)  For example, in 2004, 113 million or so folks voted in House races.  Rs took 49.2% of the aggregate; Ds took 46.6%.  So, the Rs aggregate vote margin was 2.6%.  The Rs have 232 seats to 203 for the Ds, for a 29 seat margin.  So a 2.6% vote margin translates into a 29 seat margin.  When you do this for the past thirty cycles, here's what you find out:
1.    Only once has a party won the aggregate vote and not controlled the House (1996, when Ds had a 0.3% vote margin but Rs controlled the House by 17 seats.)
2.    On average the ratio of the seat margin to the vote margin is 11.1.  I.e., a one percent margin in the aggregate voting translates into about 11 seats.
3.    If you use the 11.1 ratio, to predict seat margin from vote margin, you can then look at how close the actual seat margin is to the predicted seat margin.  For example, in 2004, we would have predicted a 29 vote seat margin (2.6 x 11.1 = 28.9).  The variance in this case was 3.  
4.    The average variance since 1946 has been 19.  So, based on this and nothing more, you could say that if the Ds take the aggregate vote by 2%, there is a 50% chance that they will have a seat margin between 3 and 41.

But, there's another way to look at the data.  We can look at the change in voting margin from one cycle to the next and ask how that translates into seat pickups.  For example, if the Ds take the aggregate vote by 2%, that would be a 4.6% increase in voting margin from 2004 to 2006.  It turns out that on average a 1% increase in voting margin translates into a pickup of 3.2 seats.  By this metric, the Ds would need a 4.7% improvement in voting margin to get an expected pick up of 15 seats.  In other words, we need a 2.1% margin in the aggregate vote to have a 50% shot at taking the House.

If that was where the story ended, I'd be a pretty happy camper.  But, in fact, the news is not quite so good.  I also looked at data on the voting margins in each House race in 2004.  These numbers are much less encouraging.  One way to cut the numbers is to rank order every district by the voting margin in 04.  Start with the Ds who got 100% and keep going until you get down to Rs who got 100%.  Since the Rs have an aggregate vote margin of 2.6%, you might think that the median vote margin - number 218 on the list - would be an R with a margin of 2% or 3%.  No.  It's an R with a margin of 10.5%.  The reason is that the Ds "waste" a lot of their votes in districts that are extremely partisan.  Some of that happens with Rs as well, just less so.  For example, 99 Ds have a vote margin over 40%.  Only 70 Rs.  This gives the Rs a significant structural advantage that is not reflected in the analysis above.  As a result, to pick up the house, Ds will have to win a lot of races that Rs won in 2004 by 8%, 9%, 10% or more.  Using the R+10.5% figure (median individual district vote margin in 2004), suggests that perhaps to take the House we need to increase our aggregate vote margin by 10.5%; i.e., that we would need an aggregate vote margin of 8%.  I don't think this can be right.  For one thing, an 8% vote margin would predict an 88 seat margin.  If in fact, it resulted in only a one seat margin, that would be a variance of 87.  The largest variance in the last sixty years was 50.  Also, I think that if we increased our aggregate vote margin by 10.5%, the increases would disproportionately show up in the more competitive districts (as opposed to places where candidates run unopposed).  In other words, I suspect that a 7% increase in aggregate vote margin would show up as 8% to 10% to 12% increases in the more competitive seats.

So, we've come up with two figures on what aggregate vote margin we need to take the House.  The first was around 2%.  The second was around 8%.  I think the truth is in the middle.  I think we need around a 5% aggregate vote margin.

Ok, suppose that's right.  How realistic is that?  Pretty realistic I think.  A poll of polls on the generic congressional polls shows Ds up by 10% or so.  I don't think we'll sustain that.  One thing I'd like to study more is how early GCPs have compared to final GCPs.  My gut is that we're likely to see that number go down fairly significantly over the summer and fall, just based on the Rs engaging in battle, spending money on advertising, stoking their base, etc.  But, the fact that we're at +10% or so now suggests that 5% or even a bit higher is very realistic.  And, by the way, see this Pew study http://people-press.org/... on the impressive accuracy of GCPs in off-year elections.

Well, I know that was a lot to read, but I'm hoping it was interesting for those of you who like to engage in this type of analysis/speculation.

•  Good stuff.

  by Stirling Newberry on Wed Jan 11, 2006 at 01:22:35 PM PDT

•  Very Interesting

  Your point about Democrats having an intrinstic structrual disadvantage because of the "packing" of their voters in certain Congressional districts (because of the urban tilt of the Democrats) is a good one.

  The analysis is great, but I wonder how much past data matters in this era of precise gerrymandering. One of the key concepts of gerrymandering is "packing", in which one party's voters are packed into a single district (typically easier to do to Democrats, as you note, but very possible to do to Republicans as well). This has always happened since the earlier days of the Republic, but computer models are sufficently advanced now, even compared to 1994, to make gerrymandering that much more effective. I have no doubt that gerrymandering after the 2000 census was the most effective of all time.

  by Hoyapaul on Wed Jan 11, 2006 at 01:30:03 PM PDT

•  This is neat.

  I think you'd get a much mroe receptive audience at mydd, though.  The faster pace in the Alito hearings will probably mean that this diary will not get the attention it normally would.  It's a shame, because you came up with a n interesting tack on giving some context for analysis of what is often a very befuddling poll number--the generic cong. ballot.  

  I would say that I appreciate you optimism, but I expect that the 8% is closer to the right answer.  That said, I 'm not sure it's an unattainable number (for the many reason I state below).  The issue with using data from the last 60 years is that it fails to capture the added redistricting precision that computer technology has allowed the GOP to achieve in order to protect their house majority.  This is the major difference between today and 1994.  So the numbers from the last decade are the only ones that are useful in terms of the precision with which GOP redistricting has distributed Dem votes to maximize their advantage (the 99 v. 70 40%+ seats you point out are a useful proxy here--was there a similar gap in, say the 74 race?). Your median race scale, on the other hand, does a much better job capturing this phenomenon.  

  I think you're right that Charlie Cook can't see the forest.  Another component that will be easier to measure in another few months is the 50-state strategy.  We'll have good Q1 and Q2 money numbers soon to determine whether the tactic is having its desired effect of soaking up GOP fundrasing dollars in new places.  If that comes to pass, then perhaps 5% will look more reasonable.  Also, I'd be interested in some form of turnour modeling or polling to attempt to analyze the extent to which 2006 will work as a base election, and whether that helps or hurts Dems.  I think either scenario would have benefits, and it's possible that the best case scenario is already underway--the engaged moderates and independent are voting against the GOP, effectively providing a stealth increase of Dem base support that may be difficult to detect before the first round of true head-to-head poll surface in the summer.  I think we'll see several more polls like Morrison's 2004 poll in June which will make a lot of GOP stomachs do back flips.

  by msbatxnyc on Wed Jan 11, 2006 at 01:31:21 PM PDT

  •  Thanks

    I just signed up for a MyDD account for the very reasons you cite.  I look at MyDD all the time, but I've never posted there.  I'll do so in this case, however.

    On your substantive comments, I agree that history is not a perfect guide ... but, I think changes tend to be marginal.  So, while redistricting has its effect, I think the historical patterns still have some relevance.  I have some nice scatter plots that help illustrate some of the points and would help show how radically inconsistent with the last 30 cycles it would be for the Dems to take aggregate voting by, say, 7%, and not take the House.  But, I don't have a web site to put the graphics on, so I left them out.

    My next step would be to try to quantify the idea that if the Dems, for example, improved their aggregate vote by 7.6% (to get to a 5% margin), that the 7.6% would show up disproportionately in the closer races.  My thought is to study a couple of elections with big changes in aggregate margin to see if indeed this disproportionaliy occurred.  Could be pretty time consuming, though!!

    One more point ... in spite of the Repubs structural advantage, their 29 seat margin is only 3 seats off from what is predicted from their 2.6% vote margin.  Go figure!

    by jay l on Wed Jan 11, 2006 at 01:40:43 PM PDT

•  recommended

  This one of the best diaries I've read in the past week. Your analysis of the aggregate numbers are interesting.

  The 1994 landside, in my opinion, is misunderstood. In the Senate that year, only two incumbent Democrats were defeated (Wofford in Pennsylvania and Sasser in Tennessee) with the remaining 6 Republican gains coming as a result of Democratic retirements. The real tidal wave that year was in the House, in which 34 Democrats were swept from office. If the American people are disappointed enough with Republican leadership, they'll take it out on someone.

  by pine on Wed Jan 11, 2006 at 01:47:31 PM PDT

•  love this

  good analysis in here.  I did find it interesting that Kos was so quick to agree with Cook's forecast: presumably he has a little more informed insight on the situation.

  I have faith in Rahm Emanuel and our slate of candidates, and particularly the Public Integrity Section of the Justice Department, to at least get very very close to taking back the House this year.

  by dday on Wed Jan 11, 2006 at 01:53:52 PM PDT

•  anti-gerrymandering as Democratic platform

  This is why I think the Democratic party should make anti-gerrymandering laws (such as Iowa has) a national platform, to be pursued in every state.  Of course, this is threatening to gerrymandered D incumbents and the usual suspects in the Beltway Establishment, but it's good politics, and it's good democracy.  

  GERRYMANDERING IS FUNDAMENTALLY UNDEMOCRATIC.  That is the lesson we should all be taking from this.  And it is flagrantly stupid for Democrats to go along with it for the sake of our incumbent's egos seats.  In the long run, it's doom.

  by Leggy Starlitz on Wed Jan 11, 2006 at 02:01:44 PM PDT

•  Packing is a serious problem for Democrats

  In urban-blue states like Illinois, Pennsylvania, and Michigan, they carve out a couple 80% Democratic districts out of Chicago, Philadelphia, and Detroit respectively.  And the rest of the state they divide into 60% Republican districts.  In the end, we get screwed.  We only finally took 10 out of 19 Illinois House seats with an upset by Melissa Bean, and this is a state that gave Kerry a 10% margin.  In Pennsylvania, we only have 7 out of 19 districts in a state that leans blue, although we thankfully have two realistic pickup chances this fall.  In Michigan, another blue-leaning state, we have an astonishing 6 of 15 districts, and no realistic pickup chances this fall.

  by Skaje on Wed Jan 11, 2006 at 02:52:40 PM PDT

•  The Bottom Line...

  ...is that in a system designed for the House to be the closest representatives of the "people," the fact that maybe 5-10% of the districts are "competitive" is an embarrassment to both parties, and our system in general, and a huge FU to our Founding Fathers.

  The system is so rigged for incumbents I believe that if the Dems don't win one chamber in 2006 they'll be in the political wilderness for a LONG time.  The idea that the GOP would retain control of ANYTHING after what they've done over the past 5 years is a truly frightening thought.

  I'm worried.  

  by jonorato on Wed Jan 11, 2006 at 03:25:03 PM PDT

•  House Races

  Very interesting analysis, I commend you for it.  While my initial reaction is that it's not quite that simple, I don't want to sound dismissive because i think you've made a real contribution here.

  Before I get into a specific reaction, I think there is a general misinterpretation of my 2006 House analysis.  About four out of five elections, this seat-by-seat, 435 race "micro-political" analysis that we at The Cook Political Report (cookpolitical.com) is known for works great, analyzing each seat's prospects individually, based on district voting patterns, canidates, campaigns, money and other factors.  

  We are the first to admit that about one out of four or five elections (most recently 1958, 1964, 1966, 1974, 1980, 1982 (House only), 1986 (Senate only) and 1994)all bets are off.  In those elections, weaker canddidates often beat stronger candidates, weak campaigns can be strong campaigns, underfunded campaigns can beat well-funded campaigns and for one party, the value of incumbency (which is huge), is somewhat devalued.  It doesnt mean that all or most of the underdogs win, but many more than normal.  We are the first to concede that.

  But this has only happened in both chambers six times in the last 50 years, and eight times that it has occurred in one or both chambers in that same half century, not withstanding many predictions that other years would be such wave elections.  We often have seen the potential for wave elections that never developed.

  So what we do is our micro-political analysis, make one prediction based off that (currently 4-6 seat gain for Dems in House, 2-4 in the Senate), then on a more narative level, discuss the prospects for an election going bigger.

  This election certainly has the potential to be everything that all you guys want it to be, but that potential is not yet reality, as just about every Democratic and Republican pollster,  media consultatn, strategist and manager we know in either party would privately admit.  It's just potential.

  Currently, notwithstanding what most of the readers of this blog fervantly hope, the public sees this as a congressional scandal involving thieving coniving politicians.  Republicans have done a very good job of muddying up the waters by pointing out the Abramoff law partners and clients who contributed to Democrats and sold the idea that "both sides got Abramoff money."  (yes, yes I know that all his personal money went to Republicans).  The investigation and likely indictment of Democratic Rep. William Jefferson doesn't help.

  For this to turn from a congressional scandal into a Republican scandal, my hunch is that at least one or two of the following things must happen to change that coloration:  DeLay is indicted on Federal charges, four or more additional Republican Members are indicted or Rove is indicted.  Then swing voters would attach a red tag to the scandals.

  Beyond the scandals, if President Bush's numbers drop back into the 30's, that would probably do it too, but if he is averaging 43 percent, and in the absence of a couple of the above factors, I don't think the wave will be especially huge given the very small number of seats in play, small number of open Republican seats in competitive districts, etc.

  Jay I 's posting was fascinating, but I had a couple of reactions.  First, his formula is based on historical data that largely was gathered in election cycles when there were far competitive districts than exist today. I've been in this business for over 30 years and have done the Cook Political Report since 1984.  In "the old days," we used to see 80, 90, 100, 120, one cycle 154 competitive districts, one out of every three in the country. Redistricting, small number of open seats, particularly in Republican-held districts that Kerry either won or lost narrowly or even ran respectably (got 46 percent or better)and anenhanced incumbency advantage most recently have diminished that to two dozen or so, three dozen max.  So the volatility and sensitivity that is built-into the historical data, applies much less so to this decade's elections.

  If there are more Jim Kolbe-like retirements, that changes, more scandals (Republican Members), that helps, the president takes another dip, that helps.

  Finally, Jay I's data is based on an assumption that if you knew what the popular vote was going to be, you could extrapolate the seat change.  While for the reasons outlined above, I don't think that is true at least in this decade with these districts and circumstances, it also assumes that you know what the popular vote breakout is going to be.

  Unfortunately, the generic congressional ballot test question on the national polls has a tendency, for some reason that no pollster has ever been able to explain, skew about five points more Democratic (actually 4.7 percent) than the popular vote.  The Hotline ran a discussion of the generic ballot test a couple of months ago and compared what the average of all the generic congressional ballot test results were for the last two weeks in each of the last four or five elections.  When you analyzed the data, what you found is that if the generic congressional ballot test showed that Dems were ahead by eight points, the House popular vote ended up about three points more Democratic than republican.  So my rule of thumb is take a look at the national generic, figure up which side has a net advantage, and subtract five points from Dems.  In recent weeks, the generic has ranged from Dems up by six (Gallup) to 13 points (AP/Ipsos), with the Hotline, Harris and Democracy Corps (Stan Greenberg, Democratic pollster) in between with 7,9 and 10 point Democratic advantages respectively.  The average is a nine point Democratic advantage.  Subtract five and you get four.

  The election isn't today, only God knows what the political climate will be in November of this year, and even using Jay I's figures, Dems theoretically need five points, they theoretically have four points today, not enough, and that still doesn't address the concerns I raised that Jay I's figures are based on a period of time that included more competitive districts and generally more open seats in play that currently exist.

  Having raised all of these issues, I still think tht Jay I has done some very thoughtful research here and look forward to reading his work in the future and see how it all pans out.

  by Charlie Cook on Sun Jan 22, 2006 at 10:27:11 AM PDT