Not Quite a Blog

The three-member panel investigating the disasterous 2006 primary in Cuyahoga County, OH have issued their report. It is here: http://cuyahogavoting.org/CERP_Final_Report_20060720.pdf

The report is amazingly detailed. And it is really scary what a little bit of sunshine will expose in a sufficiently complicated local election administration.

Here are choice nuggets I have come across... I'll be posting comments around these in the next hour or so:

---

Apparently, the numerous places that vote data were stored on decreased the care in which data was handled:

5.19 Finding: The numerous media on which voting data were stored (DRE memory cards, zero cards, CCBOE central computer, DRE internal memory, VVPAT paper rolls) led to general confusion of poll workers and CCBOE staff at to which medium carried the official votes. No single medium took on the “precious cargo” status of the paper ballots and ballot boxes of old. [Interviews with IS, Ballot Department, and pink room staff, poll workers]

Here, they wanted to record absentee votes on DREs, but the OH SoS office instructed them not to produce a paper trail (and to do this, they had to load the paper rolls backwards):

6.4 Finding: Although the DRE units are designed to function only with the VVPAT printer units properly installed, and the DRE printer could have created a paper trail recording the votes, the SOS instructed the CCBOE not to permit the DREs to print the paper receipt. The CCBOE accomplished that instruction by loading the paper backwards so that the printers did not have a surface on which they could print.

The CCBOE did not agree with the SOS's decision, which it had given orally, to have the DREs set up so they were unable to create a VVPAT.

Lost memory cards are dangerous. If someone did use a Hursti II-like hack to modify vote totals, disappearing a memory card would be a great way to hide the evidence.

6.20 Finding: As of 6/22/2006, 51 days after the May Primary election, the CCBOE has yet to recover 12 lost memory cards. Missing cards did not result in votes going uncounted. When these cards could not be found, substitute memory cards were prepared and used to collect vote data from the DREs in question, which were then uploaded and included in tabulated results.

Here we see that people didn't realize they should look at the VVPAT due to the TSx's design where an opaque cover is placed over the VVPAT window.

2.70 Finding: Many voters did not know that they could check the printer display for verification of their vote because the DRE printer’s opaque cover had to be opened in order to see the VVPAT print of their votes. If voters are not well informed about the VVPAT they will not be able to use it to verify their vote. The VVPAT display was very difficult to read for some voters because of the font size/ type and the magnifying plastic cover. The CCBOE was aware that some voters might not check the VVPAT and states that “A voter is not going to know why it is even there unless they open it up. The cover should either be eliminated or in the open position all of the time.”

Because many didn't look at the VVPAT, a paper jam resulted in multiple VVPATs printing over each other, thereby invalidating the official record of the vote.

6.52 Finding: Deputy Director Dillingham indicated that the Voter Verified Paper Audit Trail (VVPAT) was the only true “official ballot” produced by the system. Ballot Department Manager Baker indicated that some VVPAT printouts from May 2nd were unusable because the paper did not advance and multiple ballots were printed on top of each other. These two statements together indicate that official ballots were inadvertently destroyed as they were being created on election night. Many voters did not look at the VVPAT printout, and this allowed long strings of unreadable ballots to print because no one was aware of the paper jam.

Here we see reliability problems that are inconsistent and evidence of buggy hardware and software.

2.48 Finding: A number of DRE units crashed, froze, or malfunctioned during boot-up or use on Election Day, an unknown number of which were returned to service without further investigation.

Dan Tokaji mentioned that the polling place he voted in had DREs arranged in a way such that you could see your neighbor voting. Looks like there were many complaints to this effect (more than 30).

2.50 Finding: As arranged in some polling places on May 2nd, DRE units did not provide enough privacy to voters.

Here's proof that the feature set in the voter registration product (DIMSnet) is largely dictated by one very large California county (Los Angeles).

"DESI staff informed the CCBOE Ballot Department Manager that this limitation [of not being able to limit addresses for voter registration to alphanumeric characters only] was not possible because of an individual residing in California with the last name “#2”."

Yikes... the same key for every DRE in the county.

2.45 Finding: The same barrel key opens all the DRE voting devices in Cuyahoga County.

Here we see "testing" of the election modem pool by sending 60 DREs home with election administration staff.

5.4 Finding: Immediately prior to the May 2nd election, the CCBOE performed the final testing of the election night data transmission procedure they had designed. The CCBOE facilities did not have a sufficient number of telephone lines to test the modem transmission procedure prior to the election. Due to this limitation, approximately 60 CCBOE employees were selected to take one or more DREs home with them for the weekend before Election Day. They were to connect their home telephone lines to upload test voting data to the CCBOE computer. This testing was problematic on a number of grounds. [Interviews with CCBOE staff; IS staff; Deputy Director Dillingham; Director Vu]

Releasing DREs to employees to take home was a troubling breach of security. It created an opportunity for tampering and raises doubts about the CCBOE’s commitment to election security and to promoting public confidence.

Follow up:

DIMSnet upgrades are dreaded events as things break badly.

2.14 Finding: DIMSnet upgrades are problematic. Upgrades have often caused DIMSnet features to stop working. DESI testing and version management of DIMSnet is very poor. At the time DESI releases an upgrade, DESI provides little information on the problems the upgrade is designed to fix or what other changes the upgrade will make to the system. These deficiencies are compounded by the fact that DESI customer support procedures require customers to adopt all upgrades.

2.17 [...] Approximately fifty-five counties in five States use DIMSnet. This means that a large number of the features in DIMSnet designed for use elsewhere are not used by Cuyahoga County. Further, some features that would be useful for Cuyahoga County will not be created because they would not be useful for other DIMSnet customers. Essentially, the DIMSnet system design and support apparatus bars customization of DIMSnet. [...]

The next finding and recommendation are particularly interesting. It talks about how voter registration tools are not certified and should be.

2.29 Finding: The DIMSnet voter registration system was never certified by the Secretary of State or the Federal Government, because neither Sovereign currently requires certification of voter registration systems. Perhaps the justification for a lack of certification standards for voter registration database system is because they are not used to record or tabulate votes. [...]

2.30 Recommendation: Although a voter registration system does not directly influence the recording and tabulation of votes, it indirectly influences the final outcome of the voting process because the system determines who is eligible to vote. The functions of voter registration systems are significant enough that they should be thoroughly analyzed before implementation. It is essential that all technologies that are used in the election process undergo rigorous certification assessments. CCBOE joined by the Cuyahoga County Commissioners and public interest groups should file a petition with the U.S. Election Assistance Commission the Ohio General Assembly and the Ohio Secretary of State requesting their creation of certification standards.

Here the panel reports misrepresentations on behalf of DESI and that the Board of Elections probably has grounds for a lawsuit.

2.42 Finding: DIMSnet and the GEMS system were not “seamlessly integrated” as suggested by DESI. DIMSnet exports candidate and precinct information that is used for ballot creation in the GEMS system. This information could not be directly imported into GEMS. When this DIMSnet data was exported precincts became disordered, candidate names were incorrect or truncated, and information was not in the correct format to be imported to GEMS. This caused the need for tedious hours of manual data entry to the GEMS server, which only permitted one keyboard user.

2.43 Finding: The CCBOE expected that this data migration process would be relatively easy; instead it cost staff weeks of extra work which severely impacted the work schedule for the May 2nd election. [...]

2.90 Recommendation: If DESI has misrepresented its products in a manner that causes significant, unforeseen costs to be assumed by Cuyahoga County, legal recourse should be considered. Our quick review suggests the county does have standing and potential grounds, but this is a matter that should be referred to the Board’s legal counsel.

Very poor chain of custody.

2.52 Finding: Procedures regulating control and chain of custody of Election Day equipment are poor and underdeveloped. Security of the entire system depends upon access to election supplies. Tamper seals were not logged. Numerous memory cards disappeared altogether or were found weeks after the election. Some voters left polling locations with voter access cards. Not all items from the polling locations were returned in the black binders. CCBOE staff was directed to take DRE units home the weekend before the election in order to test the transmission process.

I don't know where this comes from. Sounds like the Unilect Patriot incident, not something that is a known problem with the TSx.

2.54 Finding: A study shows that voting systems may overwrite voting data when the memory card is full. [The GAO’s 11/21/2005 report]

VVPAT is useless without audits (and a nod to the Brennan Center report).

2.74 Recommendation: The CCBOE should conduct automatic routine, random audits of VVPATS according to standards that are announced and vetted in a public process. Audits are recommended by the Brennan Center for Justice’s 6/27/2006 report on voting machine security. Further, the Center includes specific guidelines for conducting a VVPAT audit in this report. Failure to conduct regular and meaningful audits of VVPATs increases the security risks greatly.

Parallel testing should be performed.

2.77 Recommendation: Cuyahoga County should perform parallel testing on a few machines during each Election Day. Additionally, the selection process of machines for parallel testing or VVPAT auditing must be transparent and random to ensure effectiveness.

Accumulation of vote data was problematic at best.

2.78 Finding: While the current transmission and accumulation practices were intended to allow for quicker tabulation of an unofficial count and allow for results to be posted at the polling place, the current practices of the CCBOE actually increase the chances for error and fraud. Further, on May 2nd, the hope of quicker tabulation through the transmission and accumulation procedure ended up taking much longer than anticipated on election night for various reasons.

Access control configuration of the GEMS server was very minimal.

2.104 Finding: In the period leading up to the May 2nd election, there were two operator accounts set up on the GEMS machine: gemsuser and gemsadmin. The former account is used for data entry and ballot configuration, the latter to configure the GEMS application itself. There was a single password for both the gemsuser and gemsadmin accounts. Considering that so many unauthorized persons were permitted to use the system and, therefore, likely to have known this password, all operators could change both the data in GEMS via the gemsuser account and reconfigure the system using the gemsadmin account. [See GAO Report#: GAO-05-956, p26] On top of this, the fact that all operators used the same, anonymous accounts (gemsuser and gemsadmin) prevents anyone examining transaction logs from determining which person made a particular modification to the system. Logs would only show that transactions were performed by either gemsuser or gemsadmin, rather than a specific individual. [Interview with Ballot Department staff; NIST SP 800-27, Rev. A: Engineering Principles for Information Technology Security (A Baseline for Achieving Security), June 2004]

There are tensions between security requirements and pratical needs.

2.107 Finding: Per Secretary of State Directive #2005-23, county election boards are prohibited from connecting GEMS to a local network, even if that network is completely isolated from other networks. It is not clear how using a single, isolated computer for all GEMS data entry is significantly more secure than a small network of dedicated computers completely isolated and not networked with other systems. This prohibition impairs the efficient use of the system.

This is a good point, too often we criticize the ITAs when it is their charge that is sub-optimal. (Although there is evidence that the ITAs have performed poorly.)

2.115 Finding: ITAs do not test, or claim to test, all potential weaknesses or vulnerabilities. Vendors and public officials often tout ITA certification as proof of reliability and security. It is not. A statement that a system is ‘ITA Certified’ is not an adequate response to public concerns about the security and reliability of an electronic voting system. Conversely, some individuals and groups wary of electronic voting criticize ITAs for not conducting tests beyond of their mandate and impute improper motives to the ITAs. This criticism is misdirected; the mandate must be changed in order to change the activities of ITAs

ITA testing has a positive impact on design. This is evidenced by the superior quality of GEMS (an ITA-tested product) compared with DIMSnet (not tested).

2.117 Finding: Whatever the shortcomings the ITA testing regime, Cuyahoga County’s experience with DESI systems offers compelling, if anecdotal, evidence that ITA testing has a positive impact on election systems. The contrasting experiences that Cuyahoga County has had with DESI’s two major election administration software products, DIMSnet and GEMS, illustrates a benefit of the ITA certification. DIMSnet, used to process voter registration, candidate, race and various other election data, does not fall into a category of systems that requires ITA certification. In Cuyahoga County’s experience, it is buggy, unstable and exhibits symptoms of poor software engineering practices. It is a monolithic, one-size-fits-all approach to handling the unique requirements of diverse electoral jurisdictions. In CCBOE’s experience, DIMSnet displays haphazard configuration management and inadequate regression testing. GEMS, used in the county for ballot creation and vote tabulation, requires certification. CCBOE has found it to be stable and well tested. It demonstrates an awareness on the part of the vendor of the crucial importance of strict configuration management. There are reasons to expect that the difference in the two systems is, in part, due to ITA testing of GEMS. The requirements of the testing, as well as the time and expense of recertification of modified systems, forces vendors to adopt sound approaches to testing and configuration management.

Once vote data is inputted into GEMS, the data ceases to be machine-specific and reverts to polling-place specific. So it is very hard to audit machine-specific results.

6.12 Finding: Auditing at the DRE level is difficult in the GEMS system. Once vote totals are uploaded into the server, the smallest detailed breakdown possible is by polling location. In other words, although GEMS can produce a report that shows each DRE’s total results, it cannot produce a report that shows the number of votes cast in the DRE for each candidate. No clear reason exists why the GEMS software could not be configured to produce a report on demand that would track and/or isolate the detailed results from a given DRE based on digital signatures. An audit program that will be used for the August election and beyond allows tracking of every Voter Access Card transaction on a given DRE, but it still does not allow a detailed breakdown of final results by machine in any GEMS output. If a security breach were to be discovered on any DRE after an election, it would be difficult to determine which ballots came from it.