So what if you don't want the NSA to see a particular file or email? If they're really wiretapping the whole goddamn internet, how can you keep secrets?
Well, you have to encrypt them. And the standard for strong encryption is PGP -- Pretty Good Privacy. PGP will lock up files and emails so that others cannot break into them.
I've heard people say recently that PGP is very hard to install, understand, and use ("Guardian reporter delayed e-mailing NSA source because crypto is a pain"). PGP -- or the open source equivalent GnuPG (Gnu Privacy Guard) -- is an encryption program, essentially like a mathematical lockbox for data and files... you can use it to encrypt files and emails in such a way that no one can get access to that information without compromising your laptop or desktop.
After hearing it was hard and having a bit of experience with it myself, I asked myself, "Just what does it take to install a working version of PGP from scratch and send email? How hard could it be?". I decided to try and see what the install process looks like installing from scratch.
(Now, of course, I'm on a Mac and will want to install a mac-specific version of it and I'll want it to work with Mac email programs like Mail.app or Mozilla Thunderbird. (Sorry, Windows and Linux folks!))
It turns out it's pretty damn hard, in general... that is, if you want to do it right, in my opinion.
There are two options: a somewhat less difficult way and a hard way:
- The more difficult way -- I describe below -- has the added benefit of being likely maintained indefinitely and being built (compiled) directly from the canonical (standard) source code for GPG.
- The easier way uses -- what appears to be -- a very snazzy piece of software maintained by a small number of volunteers, GPG Tools. (I don't have anything against GPG Tools -- I use it myself for a number of things -- but I'm wary in crypto of using boutique tools not everyone else is using and relying on implementations that could disappear. I also prefer for this kind of security tool to build it from the source code that is being actively maintained and that everyone else is using.)
If you don't share my concerns with the uniqueness and dependence of GPG Tools, I very much recommend you simply install it and follow their getting started guide. GPG Tools is open source and easily uninstalled, and I bet even PGP/GPG vets will find something to love with it's keychain manager and Services integration... and it even installs non-destructively over existing MacPorts and Fink GPG installs (more about what those are next).
If you go this route, you'll still want to, at least, read the material I highlight in step 6 below; even with working encryption software, you will still need to know a bit about what encryption is and how to properly use it. Be sure to install it from the official location (check the SSL lock!) and check the SHA1 signature of the file (something like:
shasum GPGTools-2013.5.20.dmgfrom the command-line).
Ok, with that... if you want your own built-from-source version of GPG that can also be used to encrypt or sign email), here's how.
First, I'm sorry to say but you'll need to become familiar with the basics of the Mac command-line used in Terminal.app. This may sound onerous but it will only pay dividends in the future as you'll probably want to tinker more. Here is a good place to start: "An Intro to Mac OS X's Command Line Interface". Why do you need this familiarity? Because the open source, freely and generally available version of PGP -- GPG -- is best available to your system as a working command-line program, instead of the alternative of a more traditional Mac-like package (the approach of GPG Tools, discussed above).
You'll need to install Xcode and the command line tools from Apple (The Command Line Tools can be downloaded from inside Xcode's Preferences or in a separate package). These tools provide all the computer programming tools a Mac developer needs to turn human-readable source code into executable computer programs.
You should install a package manager like MacPorts or Fink. Both of these programs are sort of like "app stores" for uber-geeky programs: they allow you to download software -- such as GPG -- that you can install directly (or indirectly by compiling/building from source code). You interact with them via the command-line or via a GUI like Pallet (for MacPorts) or FinkCommander (for Fink). Most people I talk to seem to prefer MacPorts as it seems to be less complex but Fink is very well maintained and stable.
After getting MacPorts or Fink installed and up and running, you should install gnupg. On MacPorts, you'd type the following at the command-line as an admin user:
sudo port install gnupg
with fink this is:
sudo fink install gnupg
It will do a bunch of downloading and then will either compile the software from source code or it will install a binary version of the software that someone else has already compiled.
You now need to understand a bit of how encryption works. I'm not saying you'll need to understand the math, but you need to have a basic understanding of what a key is, what different types of keys are (symmetric/asymmetric), what signing is, how to verify a signature, how to share your key, how to protect your private key, and how to encrypt/decrypt files. I'd suggest you read, in this order, sections 1, 3, 4, and 5 of the GnuPG Mini-HOWTO manual.
Now, you should be able to encrypt and decrypt files and text. To get encryption in your email program, get GPGMail for Mail.app (it's installed with GPG Tools, so you may already have it!) or the Enigmail extension for the Mozilla Thunderbird mail client.
Send me an encrypted, signed email: https://josephhall.org/gpg-key.
I have to apologize to China, although if you're Chinese proper, I doubt you'll see this apology unless you're on vacation somewhere else or using a proxy.
I've noticed a steady uptick in traffic to this site, notably this blog, over the past few months:
After some sleuthing, it seems I'm pushing 15-20 GB of traffic to China alone each month! With something like 564 million Chinese on the internet (42% penetration), I guess this shouldn't be surprising. And, yes, it was all legitimate HTTP/HTTPS traffic, mostly to this blog.
As I can't afford to pay for extra bandwith, my only option was to throttle traffic to China (e.g., using this very handy .htaccess method -- for Apache webservers -- and list of Chinese IP addresses).
That seems to have worked:
I feel bad that I have to use such a blunt tool as this blocking maneuver, but I'm not sure what else to do. Please let me know if, for some reason, this impacts you (I cannot imagine how it would).
This past week I was at the kick-off meeting of the LA County Voting System Assessment Project's (VSAP) Technical Advisory Committee. The VSAP is Registrar/Clerk Dean Logan's intense and groundbreaking effort to design, develop, procure and implement a publicly owned voting system. I am honored to be asked to serve on such an important body.
LA County is the largest election jurisdiction in the US, with 5 million registered voters, 10 languages, 5,000 precincts and a very large physical area. The county currently uses the InkaVote Plus voting system (with Audio Ballot Booth for accessibility), which is essentially an overhaul of former punchcard equipment to use inked styluses for marking and to provide in-precint checks for the voter in case they make mistakes.
Here is the InkaVote Plus system and the Microcomputer Tally System (MTS) that is used to rapidly count (>1,200 ballots per minute!) ballots after they're returned to the Registrar's Norwalk, CA headquarters:
My fellow committee members include:
- Henry Balta (Senior Assoc. CIO, LA County)
- Mike Byrne (Professor of Psychology and Computer Science, Rice)
- Josh Franklin (IT Specialist, NIST)
- Diane Goldin (Policy Coordinator, AATAP)
- Joseph Lorenzo Hall (Senior Staff Technologist, CDT)
- Brian Hancock (Director, Testing and Certification, EAC)
- Jared Marcotte (Technology Manager, Pew)
- Noel Runyan (Principal, Personal Data Systems)
- Rich Sánchez (CIO, LA County)
- Pam Smith (President, VVF)
- Charles Stewart III, (Professor of Political Science, MIT)
- David Wagner (Professor of Computer Science, UC Berkeley)
The mission of the TAC is to provide technical advice to LA County through a design and development process to meet a variety of goals and principles that LA County has determined its voting system must meet.
While we'll have an official web page and other materials soon for public perusal, I was able to take a number of photos and videos during a tour of the tabulation and storage facilities that we had during the end of the day. Find them at this Flickr photo set.
I'll leave you with the following video, that shows just how fast their card reader tabulation system, the MTS, can count ballots -- a blazing 600 ballot cards in 28 seconds! This is just one example of a metric that will be difficult to match in a new system!
Some of you close to me will know that I spent a brief period of unpleasantness trying to do good work on Wikipedia when I was in graduate school. I haven't made serious substantive edits to Wikipedia since that time.
The "high-water mark" of that period was when I started to try and moderate some Wikipedia disputes as part of the mediation cabal. I've always been a diplomat and have a passion for helping folks come to a common understanding. One instance particularly stood out as unpleasant: it resulted in someone I can only term a "wacko" writing a 60-page letter to the Dean of my program, AnnaLee (Anno) Saxenian, about alleged misconduct on my part as a mediator. Anno, of course, paid zero heed to it as she's familiar with my integrity and diplomatic tendencies.
Fast forward to this weekend, I received a communication from the person who felt wronged, almost 7 years later (and wrote the 60-page letter to Anno). He had the gall to reinstate a demand for monetary compensation (I forget why).
Anyway, in the interest of not feeding trolls but holding them up, I give you Thomas Cool, likely still indefinitely banned from Wikipedia:
-------- Original Message --------
Subject: W.r.t. democracy
Date: Mon, 18 Feb 2013 09:51:24 +0100
From: Thomas Cool / Thomas Colignatus
Dear dr. Hall,
You may recall my person from an unhappy situation at wikipedia in 2006. Someone asked me what I thought about the current entry on Arrow's theorem, and this caused me to update my comments on that. It appears that wikipedia has been misleading its readers since 2006. That update of my comments is in English as a supplement to a Dutch text:
I think that it is fair to apply 4% interest over the seven years since 2006. The sum of $150 turns into $197. I use paypal at email@example.com (still the same). My advice is that you also buy a copy of Mathematica and a license of my Economics Pack (a professional licence now is $99), so that you can start studying on the problem.
Allow me to congratulate you on completing your Ph.D. thesis. My hope is that you are better aware of scientific integrity now. Nevertheless, allow me to advise you to discuss the issue with your director Leslie Harris, just to be sure. We must hope that a Center for Democracy & Technology really understands what democracy is about.
Thomas Cool / Thomas Colignatus Econometrician and teacher of mathematics Scheveningen, Holland http://econpapers.hhs.se/RAS/pco170.htm
PM. The other relevant links:
Please note that I switched from dataweb.nl to this different location, but otherwise the links are the same.
Outlook should die, die, die.
Short version: if you use Thunderbird to read email and you regularly get
meeting.ics files from people, install Show All Body Parts in order to open/save them.
Longer version: Lately, I've had the following problem: Using Mozilla's Thunderbird email client (v17.0.2), when I receive a meeting reminder/calendar entry from a co-worker --
meeting.ics -- no matter what I do, I cannot seem to save the attached file. When I attempt to "open" it, Thunderbird responds with an error and says there's no file and when I "save" it, no file is saved. Nothing. Frustrating.
Up until today, I had been able to reasonably cope with this by looking at the raw email message source ("View" -> "Message Source"), which includes the
.ics file in text at the very end.
It's not pretty; it looks something like:
--------------030809050301030806060808 Content-Type: text/calendar; name="invite.ics" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="invite.ics" BEGIN:VCALENDAR PRODID:-//Google Inc//Google Calendar 70.9054//EN VERSION:2.0 CALSCALE:GREGORIAN METHOD:REQUEST BEGIN:VTIMEZONE TZID:America/Los_Angeles X-LIC-LOCATION:America/Los_Angeles ...
Not ideal, but oh well. At least I can read it.
However, today, I did that and saw this (!!!):
--_000_C6E91EC3EACD3E44B2A8242C2663C396027136FD09exmb2phlpewpe_ Content-class: urn:content-classes:calendarmessage Content-Type: text/calendar; charset="utf-8"; method="REQUEST"; component="VEVENT"; name="meeting.ics" Content-Transfer-Encoding: base64 QkVHSU46VkNBTEVOREFSDQpNRVRIT0Q6UkVRVUVTVA0KUFJPRElEOk1pY3Jvc29mdCBFeGNoYW5n ZSBTZXJ2ZXIgMjAwNw0KVkVSU0lPTjoyLjANCkJFR0lOOlZUSU1FWk9ORQ0KVFpJRDpFYXN0ZXJu ...
OMG. That's not readable. It's our old friend base64 encoding.
Sigh. It's not hard to decode base64 and any good techie should be able to do it... but this is my email and why should I have to write a script or something just to read email!?
So, I dug a bit deeper.
It turns out this is a "feature" of MSFT's Exchange/Outlook product line. That is, the MIME type they use for this is
multipart/alternative which means, "here are a few different versions of this email, pick whichever one you want, they are the same". This is useful if you want to send a plain text version of an email and also an HTML version of an email that looks prettier and includes clickable links and such.
However, Outlook and Exchange include
text/calendar... and -- here's the kicker -- not all the stuff in the
text/calendar version is included in the other two! And to make things totally ridiculous, Outlook transforms email into base64 encoding!!! So, you can end up exactly with the situation I had today: I had a calendar invite but no indication from the body of the email the date and time of the meeting, and when I viewed the message source, it was unreadable without decoding from base64.
Turns out, the Thunderbird extension called Show All Body Parts will display all the alternative message formats when you ask it to ("View" -> "Message Body As" -> "All Body Parts").
BTW, this is a 3.5 year old bug: Bug 505024.