New Memo from ES&S on FL CD-13 Source Code Review

UPDATE [2007-03-27T11:03:06]: David Wagner (and other team members) have pointed out that no one on the review team saw this letter until it appeared on the web and that it in no way restricted what they were allowed to write. The official statement from the team about this letter is here: http://www.cs.fsu.edu/~yasinsac/TeamComment.pdf

Kim Zetter has a blog! (It's here: Informed Consent.)

Via BradBlog, I see that Kim recently posted the text of a 3-page memo from ES&S concerning the source code review in the contested FL CD-13 race ("Source Says Second ES&S Letter Tried to Dictate What Florida Test Reports Could Say").

This letter is interesting in that it seems to set a chilled atmosphere, one in which it would be difficult to accomplish a robust source code review. Particularly, there are a set of limitations requested by ES&S that would seem to limit what the analysis team could say about their results, opinions and findings.

So, to what extent does the SAIT report -- the report of the source code review team -- comport with the restrictions and the guidelines outlined in this letter? Read on for more...

Follow up:

First, here's the list from the ES&S letter:

• No statements about possible "vulnerabilities"
• No statements about the "style" of the source code
• No statements commenting on the use of less desirable techniques, instructions, or constructs
• No statements regarding conformance to source code standards of any type or kind
• No statements regarding ES&S hardware or software engineering practices or design methods
• No statements regarding the use of preferred or non-preferred data structures, data types, data formats, databases, storage methods
• No statements rendering opinions on security techniques employed or not employed
• No statements discussing presence or absence of cryptography or other security methods and techniques
• No statements discussing relevance of any discoveries made in this review to any elections or contests outside the 2006 Sarasota General Election, U.S. House of Representative District 13 race
• No statements discussing the use or manner in which system passwords are used, constructed or stored.
• No statements regarding conformance to source code standards of any type or kind.
• No statements regarding use of desirable or undesirable style, techniques, instructions, logic, architecture, or implementation.
• No statements regarding the use of preferred or non-preferred data structures, data types, data formats, databases, storage methods.
• No statements regarding the fit and applicability of electronic technologies or this class of voting machines (i.e., Direct Recording Electronic, Touch Screen Systems) for elections in general.
• No statements regarding ES&S hardware or software engineering practices or design methods.

There are definitely parts of the report that are clearly in line with the letter, (page 4):

We do not claim that these results extend beyond the scope of our investigation. We emphasize that these findings are neither an endorsement nor a repudiation of the iVotronic, the larger class of Direct Recording Equipment (DRE) systems, nor any other form of electronic voting system. We specifically do not contend that these systems are correct or secure beyond the specific opinions that we give herein. This report is concerned solely with the question posed to us regarding the cause of the CD13 undervote in Sarasota County in November, 2006, and we do not claim that these results extend to a broader context.

However, there are other parts of the report that very much contradict one or more of the "limitations on statements" outlined in the letter. For example, there are whole sections and an entire appendix (Appendix D) on the poor use of passwords in the iVotronic software. These parts of the report go into detail about the "poor practice" surrounding passwords such as hard-coding of passwords in the source code and a special undocumented backdoor, relatively easy to exploit, that allows bypassing all password checks.

Here are the relevant passages from the last page of the public SAIT report that discuss the poor practice of passwords:

Each of the other passwords mentioned above is fixed and hard-coded into the source code. They are the same for all iVotronic machines in the country, and likely to be known to every election official who manages elections on an iVotronic machine. They can never be changed, without changing the firmware on the iVotronic machine. This represents poor practice.

The Service Menu password, Clear and Test password, ECA password, and Upload Firmware password are three-letter case-insensitive passwords. Each one is chosen to be mnemonic and easy to remember. The problem is they are also likely to be fairly easy to guess. They follow a memorable pattern. Someone who knows one of these passwords can probably guess what the other ones are without too much difficulty. These passwords provide very little security. [...]

Our judgment is that the password mechanisms on the iVotronic are poorly conceived and poorly implemented. The consequence is that the passwords by themselves do not do a good job of preventing unauthorized individuals from accessing critical system functions.

Finally, these passwords can all be bypassed using a special type of PEB, called a Factory Test PEB. When a PEB is inserted, the iVotronic machine queries the PEB to ask it what kind of PEB it is, and the PEB returns a single byte indicating what type of PEB it is. A Factory Test PEB identifies itself by returning a special single-byte value. This special value is hard-coded into the iVotronic code. Anyone who knows the special single-byte value, has access to a PEB and is able to program the PEB could construct a PEB that identifies itself as a Factory Test PEB. When a Factory Test PEB is present, all password checks are bypassed: in places where the user would normally need to enter a password, the password check is bypassed, the machine functions as though the correct password had been entered, and a log entry is appended to the event log as though the user entered the correct password. This undocumented backdoor poses a risk of unauthorized access to critical system functions, because it provides a way that a malicious individual could bypass the password checks by tampering with a PEB.

Below, in an appendix to this post, I've listed a number of other passages and areas of the report that implicate the above list. In short, there's a lot of evidence that the report authors didn't follow this agreement to the letter.

However, did the report follow any of the elements in this list of ES&S limitations? Well, they didn't talk about cryptography other than to point out that the hash values the iVotronic uses on PEBs are not keyed (which means they're easily forged). They didn't really talk about other races or other counties other than to compare similar undervote rates and how a virus attacker would have to target a number of counties to pull off certain styles of attacks. They didn't really talk about source code standards other than to talk about the structure of the source code and how hardware interrupts can lead to some unique classes of problems. They also didn't talk about the "fitness" of DREs for elections, other than to note some advantages of a paper trail in the audit context. Finally, they didn't talk at all about ES&S engineering practices or design methods. Everything else, they talked about.

The leads me to believe that this letter was part of some sort of negotiation. What the State and ES&S finally agreed to probably looks different. If this is really what the State agreed to, then I'm surprised we haven't heard something from ES&S, unless they feel that their voting system was vindicated through the review process and any attempt to enforce this agreement would be bad form.

Appendix

Here are some other areas where the report infringes upon the limitations of the letter:

Discussing the coding style of the source code as being non-ideal with respect to readability, maintainability and lack of modularization (page 16-17):

When we received the software, we did not expect to see high assurance source code. While the code meets the target 1990 voting system standard, there is a wide variation in naming and other readability characteristics. Global variables are integral components of virtually every function. While developers did not use “gotos”, control flow is not standardized and is often unintuitive. The code base is aging and shows the effects of numerous modifications. The team was frustrated by the code's limited readability, and we suspected corresponding reliability issues.

Other aspects of the code structure present hurdles for readability and maintenance, so errors could easily be introduced during updates to the code made as part of the normal software life cycle. There is an excessive reliance on global variables compounded by a lack of a high-level design to model the software components and functions. This led to a repetitive coding style, in which functions sometimes repeat checks and initializations that were performed at earlier points. We identified several benign, harmless defects caused by this strategy.

[...] Though the iVotronic code is not well modularized, it also does not suffer from well-known complexities associated with modern object-oriented programming, such as the fragile base class problem.

From the same part of the report, speculation as to the possibility of errors:

There are hardware interrupt handlers that interact with I/O devices primarily to update global variables. Those global variables are read, and sometimes also updated, by the main program, thus there is a potential for timing-dependent errors.

On page 22, the report authors talk about the follies of using a special character in the name field for a contest to allow or disallow voting in that contest:

Potential Remedy. Utilizing a special character in a name field is non-intuitive and error-prone. The contest record should include properly named and typed fields that reflect the contest's status on the ballot. It should be controlled by well-defined, clearly identified mechanisms.

When discussing the Mebane and Dill paper on page 26, the report notes that the different presentation of the ballot on accessible versus non-accessible machines could have caused a "situation" in which the undervote rate was different between these classes of machines.

Consequently, the study's results could be alternately described as revealing that ADA terminals were subject to a slightly higher-than-average CD13 undervote rate than non-ADA terminals. It is not clear why this might be so, though one possible explanation might lie with the slight differences in ballot presentation between ADA and non-ADA machines (e.g., non-ADA machines display parts of the ballot in color, while ADA machines display the ballot entirely in black and white when used by non-ADA voters).

On page 32, the report authors note that they see a possibility for an infinite recursion -- where an instruction is run repeatedly, for ever. This is definitely a discussion of a "potential situation".

The routine that records events takes a parameter indicating the event to be logged. The record is constructed and added to the event queue and is written to the event log in terminal flash. If the event queue is full, an “emergency close” routine is called. That routine immediately calls the routine that records events, causing an infinite recursion. The calling stack would grow until a hardware fault occurs. This could overwrite much of memory. It would undoubtedly cause the terminal to crash, freeze, lock up, or cease operating properly.

Section 7 on pages 36-45 goes into an extended discussion of security problems including the possibility of virus propagation. This whole section implicates ES&S limitations on discussing software vulnerabilities, opinion on security techniques and potential "situations".

On page 38, they talk about how trusting the contents of the election definition file is not justified; they later mention how the code should not implicitly trust untrusted inputs and outputs.

In the iVotronic firmware, the software implicitly trusts that the election definition file in the terminal's flash memory was generated by a legitimate entity. This assumption is not universally justified.

On page 38, more discussion of vulnerabilities:

The two prospective removable media on the iVotronic each have software security vulnerabilities.