# Analysis of a recent election

On November 18, 2017, a by-election was held in my suburb of Northcote, on account of the death by cancer of the sitting member. It turned into a two-way contest between Labor (who had held the seat since its inception in 1927), and the Greens, who are making big inroads into the inner city. The Greens candidate won, much to Labor's surprise. As I played a small part in this election, I had some interest in its result. And so I thought I'd experiment with the results and see how close the result was, and what other voting systems might have produced.

In Australia, the voting method used for almost all lower house elections (state and federal), is Instant Runoff Voting, also known as the Alternative Vote, and known locally as the "preferential method". Each voter must number the candidates sequentially starting from 1. All boxes must be filled in (except the last); no numbers can be repeated or missed. In Northcote there were 12 candidates, and so each voter had to number the boxes from 1 to 12 (or 1 to 11); any vote without those numbers is invalid and can't be counted. Such votes are known as "informal". Ballots are distributed according to first preferences. If no candidate has obtained an absolute majority, then the candidate with the lowest count is eliminated, and all those ballots distributed according to their second preferences. This continues through as many redistributions as necessary until one candidate ends up with an absolute majority of ballots. So at any stage the candidate with the lowest number of ballots is eliminated, and those ballots redistributed to the remaining candidates on the basis of the highest preferences. As voting systems go it's not the worst, although it has many faults. However, it is too entrenched in Australian political life for change to be likely.

Each candidate had prepared a How to Vote card, listing the order of candidates they saw as being most likely to ensure a good result for themselves. In fact there is no requirement for any voter to follow a How to Vote card, but most voters do. For this reason the ordering of candidates on these cards is taken very seriously, and one of the less savoury aspects of Australian politics is backroom "preference deals", where parties will wheel and deal to ensure best possible preference positions on other How to Vote cards.

Here are the 12 candidates and their political parties, in the order as listed on the ballots:

Attention: The internal data of table "4" is corrupted!

For this election the How to Vote cards can be seen at the ABC news site. The only candidate not to provide a full ordered list was Joseph Toscano, who simple advised people to number his square 1, and the other squares in any order they liked, along with a recommendation for people to number Lidia Thorpe 2.

As I don't have a complete list of all possible ballots with their orderings and numbers, I'm going to make the following assumptions:

- Every voter followed the How to Vote card of their preferred candidate exactly.
- Joseph Toscano's preference ordering is: 3,4,2,5,6,7,8,9,1,10,11,12 (This gives Toscano 1; Thorpe 2; and puts the numbers 3 -- 12 in order in the remaining spaces).

These assumptions are necessarily crude, and don't reflect the nuances of the election. But as we'll see they end up providing a remarkably close fit with the final results.

For the exploration of the voting data I'll use Python, and so here is all the How to Vote information as a dictionary:

```
1In [ ]: htv = dict()
2 htv['Hayward']=[1,10,7,6,8,5,12,11,3,2,4,9]
3 htv['Sanaghan']=[3,1,2,5,6,7,8,9,10,11,12,4]
4 htv['Thorpe']=[6,9,1,3,10,8,12,2,7,4,5,11]
5 htv['Lenk']=[7,8,3,1,5,11,12,2,9,4,6,10]
6 htv['Chipp']=[10,12,4,5,1,6,7,3,11,9,2,8]
7 htv['Cooper']=[5,12,8,6,2,1,7,3,11,9,10,4]
8 htv['Rossiter']=[6,12,9,11,2,7,1,5,8,10,3,4]
9 htv['Burns']=[10,12,5,3,2,4,6,1,11,9,8,7]
10 htv['Toscano']=[3,4,2,5,6,7,8,9,1,10,11,12]
11 htv['Edwards']=[2,10,4,3,8,9,12,6,5,1,7,11]
12 htv['Spirovska']=[2,12,3,7,4,5,6,8,10,9,1,11]
13 htv['Fontana']=[2,3,4,5,6,7,8,9,10,11,12,1]
14
15 In [ ]: cands = list(htv.keys())
```

voting took place at different voting centres (also known as "booths"), and the first preferences for each candidate at each booth can be found at the Victorian Electoral Commission. I copied this information into a spreadsheet and saved it as a CSV file. I then used the data analysis library pandas to read it in as a DataFrame:

```
1In [ ]: import pandas as pd
2 firstprefs = pd.read_csv('northcote_results.csv')
3 firsts = firstprefs.loc[:,'Hayward':'Fontana'].sum(axis=0)
4 firsts
5
6Out[ ]:
7Hayward 354
8Sanaghan 208
9Thorpe 16254
10Lenk 770
11Chipp 1149
12Cooper 433
13Rossiter 1493
14Burns 12721
15Toscano 329
16Edwards 154
17Spirovska 214
18Fontana 1857
19dtype: int64
```

As Thorpe has more votes than any other candidate, then by the voting system of simple plurality (or First Past The Post) she would win. This system is used in the USA, and is possibly the worst of all systems for more than two candidates.

## Checking IRV

So let's first check how IRV works, with a little program that starts with a dictionary and first preferences of each candidate. Recall our simplifying assumption that all voters vote according to the How to Vote cards, which means that when a candidate is eliminated, all those votes will go to just one other remaining candidate. In practice, of course, those ballots would be redistributed across a number of candidates.

Here's a simple program to manage this version of IRV:

```
1def IRV(votes):
2 # performs an IRV simulation on a list of first preferences: at each stage
3 # deleting the candidate with the lowest current score, and distributing
4 # that candidates votes to the highest remaining candidate
5 vote_counts = votes.copy()
6 for i in range(10):
7 m = min(vote_counts.items(), key = lambda x: x[1])
8 ind = next(j for j in range(2,11) if cands[htv[m[0]].index(j)] in vote_counts)
9 c = cands[htv[m[0]].index(ind)]
10 vote_counts += m[1]
11 del(vote_counts[m[0]])
12 return(vote_counts)
```

We could make this code a little more efficient by stopping when any candidate has amassed over 50% pf the votes. But for simplicity we'll eliminate 10 of the 12 candidates, so it will be perfectly clear who has won. Let's try it out:

```
1In [ ]: IRV(firsts)
2 Out[ ]:
3 Thorpe 18648
4 Burns 17288
5 dtype: int64
```

Note that this is very close to the results listed on the VEC site:

```
1Thorpe: 18380
2Burns: 14410
3Fontana: 3298
```

At this stage it doesn't matter where Fontana's votes go (in fact they would go to Burns), as Thorpe already has a majority. But the result we obtained above with our simplifying assumptions gives very similar values.

Now lets see what happens if we work through each booth independently:

```
1In [ ]: finals = {'Thorpe':0,'Burns':0}
2
3In [ ]: for i in firstprefs.index:
4 ...: booth = dict(firstprefs.loc[i,'Hayward':'Fontana'])
5 ...: f = IRV(booth)
6 ...: finals['Thorpe'] += f['Thorpe']
7 ...: finals['Burns'] += f['Burns']
8 ...: print(firstprefs.loc[i,'Booth'],': ',f)
9 ...:
10Alphington : {'Thorpe': 524, 'Burns': 545}
11Alphington North : {'Thorpe': 408, 'Burns': 485}
12Bell : {'Thorpe': 1263, 'Burns': 893}
13Croxton : {'Thorpe': 950, 'Burns': 668}
14Darebin Parklands : {'Thorpe': 180, 'Burns': 204}
15Fairfield : {'Thorpe': 925, 'Burns': 742}
16Northcote : {'Thorpe': 1043, 'Burns': 875}
17Northcote North : {'Thorpe': 1044, 'Burns': 1012}
18Northcote South : {'Thorpe': 1392, 'Burns': 1137}
19Preston South : {'Thorpe': 677, 'Burns': 639}
20Thornbury : {'Thorpe': 1158, 'Burns': 864}
21Thornbury East : {'Thorpe': 1052, 'Burns': 804}
22Thornbury South : {'Thorpe': 1310, 'Burns': 1052}
23Westgarth : {'Thorpe': 969, 'Burns': 536}
24Postal Votes : {'Thorpe': 1509, 'Burns': 2262}
25Early Votes : {'Thorpe': 5282, 'Burns': 3532}
26
27In [ ]: finals
28Out[ ]: {'Burns': 16250, 'Thorpe': 19686}
```

Note again that the results are surprisingly close to the "two-party preferred" results as reported again on the VEC site. This adds weight to the notion that our assumptions, although crude, do in fact provide a reasonable way of experimenting with the election results.

## Borda counts

These are named for Jean Charles de Borda (1733 -- 1799) an early voting theorist. The idea is to weight all the preferences, so that a preference of 1 has a higher weighting that a preference of 2, and so on. All the weights are added, and the candidate with the greatest total is deemed to be the winner. With candidates, there are different methods of determining weighting; probably the most popular is a simple linear weighting, so that a preference of is weighted as . This gives weightings from down to zero. Alternatively a weighting of can be used, which gives weights of down to

- Both are equivalent in determining a winner. Another possible

weighting is .

Here's a program to compute Borda counts, again with our simplification:

```
1def borda(x): # x is 0 or 1
2 borda_count = dict()
3 for c in cands:
4 borda_count=0.0
5 for c in cands:
6 v = firsts # number of 1st pref votes for candidate c
7 for i in range(1,13):
8 appr = cands[htv.index(i)] # the candidate against position i on c htv card
9 if x==0:
10 borda_count[appr] += v/i
11 else:
12 borda_count[appr] += v*(11-i)
13 if x==0:
14 for k, val in borda_count.items():
15 borda_count[k] = float("{:.2f}".format(val))
16 else:
17 for k, val in borda_count.items():
18 borda_count[k] = int(val)
19 return(borda_count)
```

Now we can run this, and to make our lives easier we'll sort the results:

```
1In [ ]: sorted(borda(1).items(), key = lambda x: x[1], reverse = True)
2Out[ ]:
3[('Burns', 308240),
4 ('Thorpe', 279392),
5 ('Lenk', 266781),
6 ('Chipp', 179179),
7 ('Cooper', 167148),
8 ('Spirovska', 165424),
9 ('Edwards', 154750),
10 ('Hayward', 136144),
11 ('Fontana', 88988),
12 ('Toscano', 80360),
13 ('Rossiter', 75583),
14 ('Sanaghan', 38555)]
15
16In [ ]: sorted(borda(0).items(), key = lambda x: x[1], reverse = True)
17Out[ ]:
18[('Burns', 22409.53),
19 ('Thorpe', 20455.29),
20 ('Lenk', 11485.73),
21 ('Chipp', 10767.9),
22 ('Spirovska', 6611.22),
23 ('Cooper', 6592.5),
24 ('Edwards', 6569.93),
25 ('Hayward', 6186.93),
26 ('Fontana', 6006.25),
27 ('Rossiter', 5635.08),
28 ('Toscano', 4600.15),
29 ('Sanaghan', 4196.47)]
```

Note that in both cases Burns has the highest output. This is in general to be expected of Borda counts: that the highest value does not necessarily correspond to the candidate which is seen as better overall. For this reason Borda counts are rarely used in modern systems, although they can be used to give a general picture of an electorate.

## Condorcet criteria

There are a vast number of voting systems which treat the vote as
simultaneous pairwise contests. For example in a three way contest,
between Alice, Bob, and Charlie the system considers the contest between
Alice and Bob, between Alice and Charlie, and between Bob and Charlie.
Each of these contests will produce a winner, and the outcome of all the
pairwise contests is used to determine the overall winner. If there is a
single person who is preferred, by a majority of voters, in each of
their pairwise contests, then that person is called a *Condorcet
winner*. This is named for the
Marquis de
Condorcet (1743 -- 1794) another early voting theorist. The *Condorcet
criterion* is one of many criteria considered appropriate for a voting
system; it says that if the ballots return a Condorcet winner, then that
winner should be chosen by the system. This is one of the faults of IRV:
that it does not necessarily return a Condorcet winner.

Let's look again at the How to Vote preferences, and the numbers of voters of each:

```
1In [ ]: htvd = pd.DataFrame(list(htv.values()),index=htv.keys(),columns=htv.keys()).transpose()
2In [ ]: htvd.loc['Firsts']=list(firsts.values)
3In [ ]: htvd
4
5Out[ ]:
6 Hayward Sanaghan Thorpe Lenk Chipp Cooper Rossiter Burns Toscano Edwards Spirovska Fontana
7Hayward 1 3 6 7 10 5 6 10 3 2 2 2
8Sanaghan 10 1 9 8 12 12 12 12 4 10 12 3
9Thorpe 7 2 1 3 4 8 9 5 2 4 3 4
10Lenk 6 5 3 1 5 6 11 3 5 3 7 5
11Chipp 8 6 10 5 1 2 2 2 6 8 4 6
12Cooper 5 7 8 11 6 1 7 4 7 9 5 7
13Rossiter 12 8 12 12 7 7 1 6 8 12 6 8
14Burns 11 9 2 2 3 3 5 1 9 6 8 9
15Toscano 3 10 7 9 11 11 8 11 1 5 10 10
16Edwards 2 11 4 4 9 9 10 9 10 1 9 11
17Spirovska 4 12 5 6 2 10 3 8 11 7 1 12
18Fontana 9 4 11 10 8 4 4 7 12 11 11 1
19Firsts 354 208 16254 770 1149 433 1493 12721 329 154 214 1857
```

Here the how to vote information is in the columns. If we look at just the first two candidates, we see that Hayward is preferred to Sanaghan by all voters except for those who voted for Sanaghan. Thus a majority (in fact, nearly all) voters preferred Hayward to Sanaghan.

For each pair of candidates, the number of voters preferring one to the other can be computed by this program:

```
1def condorcet():
2 condorcet_table = pd.DataFrame(columns=cands,index=cands).fillna(0)
3 for c in cands:
4 hc = htv
5 for i in range(12):
6 for j in range(12):
7 if hc[i] < hc[j]:
8 condorcet_table.loc[cands[i],cands[j]] += firsts
9 return(condorcet_table)
```

We can see the results of this program:

```
1In [ ]: ct = condorcet(); ct
2Out[ ]:
3 Hayward Sanaghan Thorpe Lenk Chipp Cooper Rossiter Burns Toscano Edwards Spirovska Fontana
4Hayward 0 35728 4505 5042 19370 21633 20573 3116 35607 4888 3335 18283
5Sanaghan 208 0 2065 2394 18648 3164 19926 2748 2835 2394 2394 17715
6Thorpe 31431 33871 0 21504 20140 20935 34010 19370 33760 35428 32726 32153
7Lenk 30894 33542 14432 0 19926 33442 34229 3886 33760 33935 32726 31945
8Chipp 16566 17288 15796 16010 0 18895 34443 6037 18845 18404 18960 33871
9Cooper 14303 32772 15001 2494 17041 0 34443 3395 18075 18404 15548 31608
10Rossiter 15363 16010 1926 1707 1493 1493 0 4101 18075 18404 17041 15906
11Burns 32820 33188 16566 32050 29899 32541 31835 0 35099 35428 32726 32024
12Toscano 329 33101 2176 2176 17091 17861 17861 837 0 3887 2902 18075
13Edwards 31048 33542 508 2001 17532 17532 17532 508 32049 0 20359 18075
14Spirovska 32601 33542 3210 3210 16976 20388 18895 3210 33034 15577 0 20717
15Fontana 17653 18221 3783 3991 2065 4328 20030 3912 17861 17861 15219 0
```

What we want to see, of course, if anybody has obtained a majority of preferences against everybody else. To do this we can find all the values greater than the majority, and add up their number. A value of 11 indicates a Condorcet winner:

```
1In [ ]: maj = firsts.sum()//2 + 1; maj
2Out[ ]: 17969
3
4In [ ]: ((ct >= maj)*1).sum(axis = 1)
5Out[ ]:
6Hayward 6.0
7Sanaghan 2.0
8Thorpe 11.0
9Lenk 9.0
10Chipp 6.0
11Cooper 5.0
12Rossiter 2.0
13Burns 10.0
14Toscano 2.0
15Edwards 5.0
16Spirovska 6.0
17Fontana 2.0
18dtype: float64
```

So in this case we do indeed have a Condorcet winner in Thorpe, and this election (at least with our simplifying assumptions) is also one in which IRV returned the Condorcet winner.

## Range and approval voting

If you go to rangevoting.org you'll find a
nspirited defense of a system called *range voting*. To vote in such a
system, each voter gives an "approval weight" for each candidate. For
example, the voter may mark off a value between 0 and 10 against each
candidate, indicating their level of approval. There is no requirement
for a voter to mark candidates differently: a voter might give all
candidates a value of 10, or of zero, or give one candidate 10 and all
the others zero. One simplified version of range voting is approval
voting, where the voter simply indicates as many or as few candidates as
she or he approves of. A voter may approve of just one candidate, or all
of them. As with range voting, the winner is the one with the maximum
number of approvals. A system where each voter approves of just one
candidate is the First Past the Post system, and as we have seen
previously, this is equivalent to simply counting only the first
preferences of our ballots.

We can't possibly know how voters may have approved of the candidates, but we can run a simple simulation: given a number between 1 and 12, suppose that each voter approves of their first preferences. Given the preferences and numbers, we can easily tally the approvals for each voter:

```
1def approvals(n):
2 # Determines the approvals result if voters took their
3 # first n preferences as approvals
4 approvals_result = dict()
5 for c in cands:
6 approvals_result = 0
7 firsts = firstprefs.loc[:,'Hayward':'Fontana'].sum(axis=0)
8 for c in cands:
9 v = firsts # number of 1st pref votes for candidate c
10 for i in range(1,n+1):
11 appr = cands[htv.index(i)] # the candidate against position i on c htv card
12 approvals_result[appr] += v
13 return(approvals_result)
```

Now we can see what happens with approvals for :

```
1In [1 ]: for i in range(1,7):
2 ...: si = sorted(approvals(i).items(),key = lambda x: x[1],reverse=True)
3 ...: print([i]+[s[0] for s in si])
4 ...:
5[1, 'Thorpe', 'Burns', 'Fontana', 'Rossiter', 'Chipp', 'Lenk', 'Cooper', 'Hayward', 'Toscano', 'Spirovska', 'Sanaghan', 'Edwards']
6[2, 'Burns', 'Thorpe', 'Chipp', 'Hayward', 'Fontana', 'Rossiter', 'Spirovska', 'Lenk', 'Edwards', 'Cooper', 'Toscano', 'Sanaghan']
7[3, 'Burns', 'Lenk', 'Thorpe', 'Chipp', 'Hayward', 'Spirovska', 'Sanaghan', 'Fontana', 'Rossiter', 'Toscano', 'Edwards', 'Cooper']
8[4, 'Burns', 'Lenk', 'Thorpe', 'Edwards', 'Chipp', 'Cooper', 'Fontana', 'Spirovska', 'Hayward', 'Sanaghan', 'Rossiter', 'Toscano']
9[5, 'Thorpe', 'Lenk', 'Burns', 'Spirovska', 'Edwards', 'Chipp', 'Cooper', 'Fontana', 'Hayward', 'Sanaghan', 'Rossiter', 'Toscano']
10[6, 'Lenk', 'Thorpe', 'Burns', 'Hayward', 'Spirovska', 'Chipp', 'Edwards', 'Cooper', 'Rossiter', 'Fontana', 'Sanaghan', 'Toscano']
```

It's remarkable, that after , the first number of approvals required for Thorpe again to win is .

## Other election methods

There are of course many many other methods of selecting a winning candidate from ordered ballots. And each of them has advantages and disadvantages. Some of the disadvantages are subtle (although important); others have glaring inadequacies, such as first past the post for more than two candidates. One such comparison table lists voting methods against standard criteria. Note that IRV -- the Australian preferential system -- is one of the very few methods to fail monotonicity. This is seen as one of the system's worst failings. You can see an example of this in an old blog post.

Rather than write our own programs, we shall simply dump our information into the Ranked-ballot voting calculator page and see what happens. First the data needs to be massaged into an appropriate form:

```
1In [ ]: for c in cands:
2 ...: st = str(firsts)+":"+c
3 ...: for i in range(2,13):
4 ...: st += ">"+cands[htv.index(i)]
5 ...: print(st)
6 ...:
7354:Hayward>Edwards>Toscano>Spirovska>Cooper>Lenk>Thorpe>Chipp>Fontana>Sanaghan>Burns>Rossiter
8208:Sanaghan>Thorpe>Hayward>Fontana>Lenk>Chipp>Cooper>Rossiter>Burns>Toscano>Edwards>Spirovska
916254:Thorpe>Burns>Lenk>Edwards>Spirovska>Hayward>Toscano>Cooper>Sanaghan>Chipp>Fontana>Rossiter
10770:Lenk>Burns>Thorpe>Edwards>Chipp>Spirovska>Hayward>Sanaghan>Toscano>Fontana>Cooper>Rossiter
111149:Chipp>Spirovska>Burns>Thorpe>Lenk>Cooper>Rossiter>Fontana>Edwards>Hayward>Toscano>Sanaghan
12433:Cooper>Chipp>Burns>Fontana>Hayward>Lenk>Rossiter>Thorpe>Edwards>Spirovska>Toscano>Sanaghan
131493:Rossiter>Chipp>Spirovska>Fontana>Burns>Hayward>Cooper>Toscano>Thorpe>Edwards>Lenk>Sanaghan
1412721:Burns>Chipp>Lenk>Cooper>Thorpe>Rossiter>Fontana>Spirovska>Edwards>Hayward>Toscano>Sanaghan
15329:Toscano>Thorpe>Hayward>Sanaghan>Lenk>Chipp>Cooper>Rossiter>Burns>Edwards>Spirovska>Fontana
16154:Edwards>Hayward>Lenk>Thorpe>Toscano>Burns>Spirovska>Chipp>Cooper>Sanaghan>Fontana>Rossiter
17214:Spirovska>Hayward>Thorpe>Chipp>Cooper>Rossiter>Lenk>Burns>Edwards>Toscano>Fontana>Sanaghan
181857:Fontana>Hayward>Sanaghan>Thorpe>Lenk>Chipp>Cooper>Rossiter>Burns>Toscano>Edwards>Spirovska
19</pre>
```

The above can be copied and pasted into the given text box. Then the page returns:

winner | method(s) |
---|---|

Thorpe | Baldwin |

Black | |

Carey | |

Coombs | |

Copeland | |

Dodgson | |

Hare | |

Nanson | |

Raynaud | |

Schulze | |

Simpson | |

Small | |

Tideman | |

Burns | Borda |

Bucklin |

You can see that Thorpe would be the winner under almost every other voting system. This indicates that Thorpe being returned by IRV seems not just an artifact of the system, but represents the genuine wishes of the electorate.