The voting crowd is emerging from a 2-day workshop with election officials on remote voting for military and overseas voters. I’m trying to get a sense of attendees’ impressions from that workshop, but suffice it to say that it seems to have been “exciting.” Ron Rivest compared online voting for public-office elections to drunk driving, as in “there’s no good way to do it,” and that apparently didn’t go over very well with some folks. I agree with the metaphor, however harsh it may seem.

Meanwhile, there is plenty of room for online voting in the numerous elections people hold that are not for public office: corporate boards, clubs, student government, etc. That’s why I’m excited that last night, we released Helios v3. Try it out right now by voting in our super geeky sample election and in a current-events election (Wyclef for Haiti?).

More than 25,000 votes have been cast using Helios technology. We’ve learned some very interesting lessons already, and there are many more to come. I’m hoping that, as we add more social aspects to Helios, we’ll see more usage, more data, and a unique chance to improve the technology based on real-world experience.

Debating voting methodology can usually get very heated. In fact, if I say anything negative about ranked-voting, more formally called instant-runoff voting (IRV), a legion of IRV fans will descend upon this blog with tremendous fury. Thankfully, in this case, there’s little room for disagreement: it’s pretty obvious that IRV will much more adequately represent the opinion of the Academy. In fact, it’s surprising that the Academy has been using plurality single voting, which can easily yield wildly inaccurate results. It makes one question the validity of past Oscar winners, and not only because the election is completely un-auditable by anyone other than the designated auditor firm.

Say, for example, that 30% like Avatar best, 25% Hurt Locker, 20% Inglorious Bastards, 15% Up in the Air, and 10% District 9. (Apologies to the other Oscar nominees, but I need a simple example.) Using last year’s voting method, Avatar wins. With 30% of the vote. But wait, what if the fans of District 9 hated Avatar, and really prefer Hurt Locker second best? Since their first choice was District 9, a less popular movie, it seems they effectively don’t have an impact on the result of the election… unless we take their second choice into account. Ok, so we give those 10% to Hurt Locker, and now Hurt Locker wins. But wait, what if the fans of Up in the Air mostly prefer Avatar to Hurt Locker, so we eliminate “Up in the Air” for not having received enough votes, then give those to Avatar, then Avatar wins, but wait… you get the picture. It’s not that complicated. Basically, it means that if the movie you really want to see win has no chance of winning, then we’ll look at your second choice instead. The really crazy thing is that, with last year’s method, it’s conceivable that, even if all the fans of Inglorious Bastards, Up in the Air, and District 9 prefer Hurt Locker to Avatar, meaning that in a 2-way-only election, Hurt Locker would win 70-30, Avatar STILL wins under the system used for the last 64 years.

Because of this oddity, the fans of District 9 might realize that their favorite has no chance and be tempted to select only between the two favorites, Avatar and Hurt Locker. In other words, the dark horses are inherently handicapped. With IRV, there’s no reason to resort to such silliness: vote for the dark horse first if that’s really your preference, and if not enough others agree, your second choice will be “activated,” and you won’t have lost your chance to influence the result. So, this year, a dark horse movie has a better chance of winning. But not because the voting system gave the dark horse an unfair advantage! Rather, because IRV better represents the will of the Academy. Even if one of the favorites does win, it will be a much more legitimate win than every year prior.

And here’s the funny thing. That crazy plurality single vote system I just described… that’s how we vote for President in the United States.

Wait a minute…

Did I just imply that IRV is awesome? I should be more careful. Everything I just explained assumes that voters are well informed and rational. I’m willing to believe that voters are mostly rational, but I don’t think they’re well informed. Specifically, a voter might easily believe that voting first for District 9, then for Avatar yields a “weaker” vote for Avatar if District 9 is knocked out of the running. Or, they might think that voting only for District 9 will yield a stronger vote than if they add a second or third choice because, in some sense, District 9 is then the only acceptable winner for those single-movie voters. In other words, I suspect voters will still vote strategically with IRV, only this time with an incorrect, ill-informed strategy. This is speculation, I don’t have hard numbers to back it up, only (significant) anecdotal experience with voters who find IRV deeply confusing.

What we really want is a voting system that assumes realistic behavior from voters who are typically not fully informed experts. In a way, we need to reduce flexibility for voters so that the average voter will be less likely to choose an ill-informed strategy. That method is probably approval voting, where a voter marks every candidate they find acceptable. No ranking, just a checkmark next to each candidate. Instructions are then very straight-forward: mark every candidate you would be happy to see win. Not perfect in terms of ill-informed-strategy-resistance, but a heck of a lot better than all the misconceptions that come with IRV.

Oscar voting is actually even weirder

Of course, as if the insanity of the Oscars’ voting system over the last few years weren’t enough, there’s more weirdness.

To select the nominees, the Oscars effectively run a multi-seat Single Transferable Vote, which is like IRV where you rank the options, but this time you’re filling multiple spots. This is the way that Cambridge, Massachusetts elects its City Council, and it’s the way Australia elects its Parliament, and it’s incredibly confusing because of how votes are redistributed when a candidate is knocked out of the running or, more importantly, how to redistribute extra votes for a candidate that already has passed the victory bar. How confusing? Well, in Cambridge, the result of the election may depend on the order in which you count the ballots. Yep, you read that right, in a close election, the order of the ballots matters.

I’m not sure how it works exactly for the Oscar nomination process, but apparently the Oscars add a second complication: a nominee must be selected as a first choice by at least one person. Even if the movie is everyone’s second choice, it cannot be a nominee.

So, what this now means is that the Oscars are using a weirdly modified version of multi-seat Single Transferable Vote to select the nominees, and then a plurality single-vote to choose among those nominees, except this year where they’re re-running an IRV vote for Best Picture.

And to top it all off, you have to fully trust PriceWaterhouseCoopers, the auditors, who don’t even provide tallies, only the name of the winners.

Whoever said elections are simple?

I particularly enjoyed the problem Loopt faced with respect to abusive spouses: if your spouse is spying on you, it’s not enough to turn off your location services, because then your abusive spouse will know that you’re hiding something. You have to actually be able to lie about your location, in other words Loopt has to let you fake your location data. And they do. And that’s awesome.

It’s just like voting: to be free to vote the way you want to vote, you have to be able to claim that you voted a certain way, even if you voted another way, and that claim has to be believable. In fact, when you think about it, because Loopt offers this “fake my data” feature, there’s no way for you to prove to someone else that you really are where you claim to be, at least not via Loopt. Because, if there were a way to say “okay, really, I’m here, no faking this time,” then there would be no deniability since abusive spouses could simply ask for the extra-no-faking version of the location.

In other words, to truly achieve deniability, you have to take away the user’s ability to certify their own data. That’s not obvious, and it’s interesting that location-based services and voting have this point in common.

You’ll find the complete instructions here on the auditing site.

I haven’t tested this on Windows, just Mac OS X, and it should work on Linux/Unix, too. You need Python 2.5 or above, PyCrypto, git, and subversion. You need about 30 minutes of download time, and 1 hour of processing. And then you can check the results you’ve computed against the results I’ve computed, against the official election results (which have some small variations since the results were certified, I’m not entirely sure why), and against the list of verification codes.

Having trouble remembering which table is which? Here’s a reminder:

Now of course we don’t actually see these tables in cleartext, rather what we have right now is:

Next, the Scantegrity team used random stock data to seed a random number generator and decide which side to open up, the left or right. Now we said before it would be row-per-row in the shuffle table… but because of a subtle privacy issue, it turns out instead that there are 40 Shuffle Tables, and each one will be open either entirely on the left, or entirely on the right.

That’s what was done, and that’s what I verified just now in the Meeting 4 Audit.

Any Issues?

The same issue that I mentioned for Meeting 2 applies: the stock-data program pulls data that is, unfortunately, still being adjusted for a few days as stock trades reconcile, and thus it’s not possible for me to find the exact same seed. I have to trust that the Scantegrity team did it right. Not ideal, as I mentioned before, but also not extremely worrisome because the other parts of the audit provide a bit of insurance against any error here: next we’re going to open up every piece of data for the unused ballots, and there are a lot of those, and no carefully crafted randomness can hide cheating here.

Contested Ballots?

Scantegrity supports a “contested ballot” audit where, if a voter wants to contest that their confirmation code does not appear, all confirmation codes for that ballot are opened up to prove there was no hanky-panky. Since that original nomenclature, the Scantegrity team has decided to effectively act like *all* ballots are contested by default, so that all confirmation codes for all cast ballots are revealed to prove that there are no duplicates or other naughtiness. Of course, the correspondence between confirmation code and real, decoded candidate is not revealed.

If you voted in the Takoma Park election, you can check your ballot’s complete set of confirmation codes against my regenerated list.

Auditing Unused Ballots

For all ballots that were unused, the Scantegrity team is now forced to reveal all of the shuffle table rows and all of the confirmation codes. Since it’s not possible for the Scantegrity team to predict ahead of time which ballots would be used and which would remain unused, this audit gives us added confidence that the used ballots were okay, too, because the unused ones are.

Check out the Unused Ballot Audit. Everything checked out fine.

So… are we done?

Yes, we are! It looks like the Takoma Park election went well, and I can say with confidence that, if you voted in the election, wrote down your confirmation code, and checked it against my copy of the confirmation codes, then your vote counted the way you intended it to. And I can say this even without knowing how you voted. Pretty darn cool.

Over the next few days, I’ll write up a couple of followups regarding:

• some recommendations for improving Scantegrity,
• an explanation of what happens if the election officials are all corrupt, and
• a script that lets you re-perform the entire Scantegrity audit on your own.

Except for one more detail: the Scantegrity team is continuing the Instant Runoff candidate elimination past the point of a candidate gaining absolute majority. I think that’s wrong. It doesn’t affect the outcome, but it does affect the final tally count, so we’ll wait and see what the official word is.

In any case… isn’t it cool that we can audit each other and work out these differences with public code, public results, and complete oversight from anyone who wants to watch? That, again, is the power of open-audit elections using systems like Scantegrity or Helios.

So the votes have been cast, and voters went home. Some of them wrote down their confirmation codes. They probably checked those codes against the official Scantegrity web site. But why would they trust that web site to do all of the math right in the backend?

That’s where the audit work comes in. I’ve now run the Meeting 3 verification, and it looks good: the confirmation codes were properly opened, and I’ve posted my own re-computed version of the confirmation codes. If you’re a Takoma Park voter and you want extra certainty that your vote counted, you should check those confirmation codes and let me know if your confirmation codes don’t appear properly.

But it’s not just the confirmation codes, since we now have the unofficial tally. I’ve posted the tally that I have re-computed from these ballots. Very close to the preliminary results from the unverified opscan software itself, as in, off by only a couple of votes here and there in no way that comes close to changing the results.

Ummmm, but you said this was verifiable, so where is the discrepancy coming from?

There is paper involved, and there is scanning of paper involved. Whenever that happens, errors will occur, either in the normal opscan process, or in the reading of verification codes. If I had to bet money, I’d say it’s probably the opscan scanning that is off, while the Scantegrity code is exactly correct. But, we probably won’t know for sure, and it doesn’t make a difference as long as very few (hopefully no) voters complain about missing confirmation codes.

Now remember, again, this is the unverified tally. We have to give voters the chance to complain about their confirmation codes first. Only then will we run the final audit steps, Meeting 4 + the spoiled ballot checks.

Can I run this myself already?

Yes, check out my audit code from github:

git clone git://github.com/benadida/scantegrity-audit.git

and do a subversion checkout of the Scantegrity data:

svn checkout https://scantegrity.org/svn/data/takoma-nov3-2009

Instructions on how to run the verifications are in the README file, in this case:

python meeting3.py {DATA_DIR} {CONFIRMATION_CODES_OUTPUT_FILE_PATH}

for each of the 6 wards’ data directory. The confirmation codes are written to the given output file.

Then, to run the tally:

python tally.py {QUESTION_ID} {DATA_PATH_1} {DATA_PATH_2} ...

For QUESTION_ID 0, the mayoral race run across all wards, you’ll need all 6 data paths.

For QUESTION_ID 1, you’ll need to run tally.py against each individual ward’s data path, since those are different races.

OK, so a couple of days ago we verified the initial P table and D tables for all 6 wards in tomorrow‘s Takoma Park election. Now comes Meeting 2, which was held a couple of weeks ago to open up a random half of those ballot commitments to ensure that the P and D tables were generated correctly.

The short version of the story is that it all checks out, and the ballots look well-formed. Check out the detailed audit data.

That said, there was one issue that might reduce one’s confidence in the validity of the cut-and-choose, and there was another issue that was annoyingly preventing me from verifying the data until just now. So, let’s get our hands dirty and see …

Where do we get random numbers?

Meeting One was held on October 12th. How do I know this? I downloaded the data on October 13th, including the list of all files produced and their SHA1 fingerprints, and I signed it on October 13th at 3pm Pacific. You can download and verify the signature for yourself (DSA key ID 0F25B7E6). How can you be sure that’s my key? Well, you might want to ask me next time you see me in person, and I can confirm that the Scantegrity team didn’t hijack my blog and keep me locked up in a dungeon somewhere to prevent me from speaking out.

On October 14th, one day later, the Scantegrity team downloaded stock data from that day, using a script they had also committed to on October 12th as part of their Meeting 1 release. They used this stock data, which anyone can publicly verify and which is very hard to predict ahead of time, to generate the set of “challenge ballots,” meaning the P and D table rows that they would be forced to open.

Problem #1: Is Stock Data ever Final?

I discovered an annoying little issue: as it turns out, Google’s stock volumes are not stable, even a few hours after market close. They eventually add after-market trades, and there are trades that reconcile later that could affect the volume numbers. Indeed, on October 14th, the Scantegrity team got the following data:

NYSE:MMM    14-Oct-09,75.35,76.93,75.07,76.57,4120300
NYSE:AA     14-Oct-09,14.36,14.38,14.21,14.32,28884785
NYSE:AXP    14-Oct-09,35.27,35.31,34.57,35.09,15329442
NYSE:T      14-Oct-09,26.22,26.25,25.78,25.83,32644760
...

and today, I got the following data:

NYSE:MMM    14-Oct-09,75.35,76.93,75.07,76.57,4121804
NYSE:AA     14-Oct-09,14.36,14.38,14.21,14.32,28920161
NYSE:AXP    14-Oct-09,35.27,35.31,34.57,35.09,15334664
NYSE:T      14-Oct-09,26.22,26.25,25.78,25.83,32660582
...

Notice how the stock prices are the same, but the volumes are slightly higher in my dataset.

Of course the Scantegrity team didn’t do anything naughty here. But let’s be paranoid for a second.

Technically, because I can’t find a way to truly verify the original Scantegrity random-data seed, it’s conceivable that each line in this seed-file could be tweaked to any one of a few thousand values without detection, and thus that the officials could have done an exhaustive search of the hash domain to audit only the ballots they generated correctly, but never the ballots they “purposely” generated incorrectly. Those hypothetical incorrectly generated ballots could be set up to flip the selections of individual ballots in a way that we cannot detect right now, and if those ballots could be handed to people who are known to vote the “wrong” way, then they could be effectively forced to vote the “right” way.

Except, of course, that since each row can be audited with probability 50%, it would be computationally very difficult for a malicious administrator to cheat on more than 50 or 60 ballots. Very, very hard. 100 ballots? For all intents and purposes, impossible with today’s computing power. And then those 50 or 60 ballots would have to be handed to people you *know* are going to vote for the candidate you’re opposing… so in the worst-case scenario, with a very powerful adversary and a significant amount of coordination, this might swing the results by a few votes…..

Except not even that, because there is still a safety net: at the end of the election: the unused ballots will be spoiled and fully revealed. Chances are, if there is any significant number of bad ballots, they will be detected then, and an investigation can begin.

So this is a bit of a weakness, but not one that can realistically enable corruption of the election without detection. And of course, that’s the point: you can never prevent corruption attempts, but with open-audit voting like Scantegrity, you can detect it.

Another little tidbit

Even after this first pass, things still weren’t checking out, so I consulted with the Scantegrity team, and together we realized that the set of challenge ballots had been generated on Windows, but when the same program and random-data seed file were run on Linux or Mac, they generated a *different* challenge set. Why? It has do with carriage-return and line-feeds, and we’re still figuring out exactly how to prevent this in the future… but the point is that, by re-adding in the carriage return characters, everything checked out.

Is this a security vulnerability? No, it’s not, since there are only two possible representations for newlines, the Windows and Mac/Linux ways, so there’s no room to squeeze in any cheating here. It’s just a bug. And here’s what’s always been interesting to me about open-audit voting: your first verification might not work, your second verification might not work, because of annoying little bugs. But you can always iron out those bugs and re-run the verification. Because after all, there might be bugs in the audit procedure, too. With open-audit voting, you can often redo the audit, and when you do get a thumbs-up, you know things are in good shape. That’s powerful.

Conclusion… so far

Meeting 2 is verified… with the caveat that, hypothetically, we can’t be 100% certain that some hanky panky didn’t happen on the randomness generation. That’s okay, though, because realistically, in the worst-case scenario we can imagine, only a small handful of ballots could be affected, and in any case we’ll regain full confidence once we run the spoiled-ballot checker at the end.

One lesson I’m drawing from this: the cut-and-choose proofs based on public randomness are very tricky to pull off, because they can’t be re-done: the ballots are printed, and the challenged ballots are discarded. We can’t go back and re-do the proof of validity of the existing ballots. Since open-audit voting systems are powerful specifically because it’s always possible to undo something bad (re-vote, re-verify the tally, etc…), I wonder if Scantegrity might benefit a bit from a different proof protocol. I don’t know what that would look like yet, though….

In any case, Meeting 2 is verified to my satisfaction.

UPDATE: want to audit the data yourself? Go check out my audit code from github:

git clone git://github.com/benadida/scantegrity-audit.git

and do a subversion checkout of the Scantegrity data:

svn checkout https://scantegrity.org/svn/data/takoma-nov3-2009

Instructions on how to run the verifications are in the README file, in particular

python meeting1.py {DATA_DIR}

and

python meeting2.py {DATA_DIR}

making sure, for that second one, that you’ve copied the djia-stock-prices-latest.txt to {DATA_DIR}/pre-election-random-data.txt, where {DATA_DIR} is one of the wards.

Each verification of a ward’s single meeting will take a couple of minutes on an average PC. This isn’t the fastest audit code ever, it’s written to be easily audited, even if that makes it a bit slower than necessary.

If you’ve been following, we know what the voter experience is going to be like on Tuesday, and we know what the auditing process is going to be like. So, can we audit this thing already?

Yes, we can. Here are the steps:

1. Meeting 1: the election officials get together, agree on election parameters, and generate the commitments to the Ballot Table of 5000 ballots (called the P Table for historical reasons) and the 40 Shuffle Tables (called the D Tables). Why 40 shuffle tables? It’s a way of increasing the certainty of the election verification, but we’ll talk about the details in a later blog post. The only thing to verify here is that the Ballot and Shuffle tables contain exactly the number of rows they’re supposed to contain.
2. Meeting 2: the election officials get together and use some public source of randomness (stock closing prices) to decide which rows in the P table (and corresponding rows in the D tables) to open up for auditing in the cut-and-choose process. They then reveal those rows accordingly. For the remaining rows, meaning the ballots that will get printed and used for ballot casting, the election officials generate commitments to the confirmation codes and publish them. The job of the auditor here is to ensure that this random selection was done properly, that the row reveals match up with the data seen in Meeting 1, and that the left-hand and right-hand permutations in the D tables match up with the P table. The confirmation code commitments are good to record, but they can’t be checked yet.
3. After the election happens, Meeting 3: the election officials get together and fill in the P tables with the encoded votes that were cast and the D tables with the intermediate decryption of these encoded votes. They also reveal the confirmation codes that voters should be able to check only, and how these confirmation codes correspond to the commitments from Meeting 2. An auditor should ensure that the ballots used here were not those audited in Meeting 2, and that the codes are revealed correctly. An auditor should also make available the list of confirmation codes that he verified.
4. the Tally: well yeah, that’s kind of important… based on the Results Table, called the R table, published in Meeting 3, it’s straight-forward to re-compute the tally from the individual ballots there. In the case of the Takoma Park election, this is a relatively standard single-seat single-transferable election.
5. Meeting 4: using public randomness again, the election officials open up either the left or right hand of each D table. The auditor must ensure that the randomness was properly used to generate the left/right challenges, that the reveals match the earlier commitments from Meeting 1, and that the revealed permutation, left-hand or right-hand, was properly applied: in the case of the left-hand permutation that the P-table vote is properly transformed into the intermediate D-table vote, in the case of the right-hand permutation that the D-table intermediate vote was properly transformed into the result, fully decrypted vote.
6. Contested Ballots: if voters complain about a confirmation-code mixup after Meeting 3, then the contested ballots are fully opened up by administrators so that all confirmation codes can be revealed. An auditor should check that these contested ballots are properly opened (we expect very few.)
7. Spoiled Ballots: in-person auditing staff will be randomly selecting ballots to audit, and all unused ballots will be audited too in this “spoiled-ballot” final check, where the full P-table row and corresponding D-table rows are revealed by election officials. The auditor should check that all reveals are done correctly and that all permutations match between the D tables and P table.

Oh yeah, and one more thing: everything has to happen 6 times because there are 6 wards, each of them run independently.

So, the election is in a couple of short days… where are we now?

Meetings 1 and 2 have occurred. And I have just audited Meeting 1, go see the Meeting 1 audit data.

I haven’t yet done the audit for Meeting 2, but I have already signed the files generated by the Takoma Scantegrity team, so that I can be certain that that data is locked and loaded. I’ll be auditing it shortly, before the Tuesday election of course.

Hold that thought.

These last few days saw a couple of fascinating announcements in the voting machine world:

• the Open-Source Digital Voting foundation released their open-source election system, and
• Sequoia, one of the established voting machine vendors, will disclose all of its source code, from voting machine to tallying system.

These are great developments. We need a clear principle: you cannot count elections in secret, and proprietary code means counting elections in secret. Disclosing source code is a great step towards transparency in election verification. Open-source from OSDV is also a great way to break the vendor lock-in on voting machines and enable true innovation in voter user-interfaces, end-to-end verification add-ons, etc. All in all, this is a hugely positive development, in particular the OSDV release.

But we also can’t fall for the hype: when I walk into the voting booth, how do I know that voting machine is running the same source code I verified from my home machine? Do my inspections of the released codebase have any bearing on the machine I’m about to use?

To be fair, yes, to a degree: the election officials’ job will likely include ensuring that the authorized code is properly installed. But….

• states and counties often customize voting machines for their specific needs. As any security guy will tell you, if you verify 99% of the code, from a security standpoint you’ve verified nothing at all. Who’s verifying all of the customizations?
• it’s very difficult to build hardware that will give you some indication that the software you installed on it wasn’t tampered with. Think rootkits/viruses for your desktop computer.
• even if no modifications are made, even if a truly trusted device is built and deployed, in the end, you still end up having to trust the election officials to do their job entirely correctly, because a corrupt open-source machine is no more reliable than a corrupt proprietary machine, and using today’s devices, it only takes one corrupt election official to corrupt a voting machine, open-source or not.

I understand that free software fans see this as a revolution in election trustworthiness. However, in the case of voting machines, while disclosing source code is an improvement, it’s not nearly the improvement that some are claiming. You might disagree, of course, and then maybe you would be willing to sign for that other house down the road, the one you never saw with you own eyes, the one that nobody you know inspected.