the consumer secret is not important

The article’s main argument is that the oAuth consumer secret is embedded in desktop clients and can be extracted. Yes. That sounds really bad doesn’t it? Except, as the article itself says:

It’s very important to understand that a compromised consumer secret key doesn’t jeopardize the security of the users of the application. The key can’t be used to gain access to the accounts of other users, because accessing an individual account requires an access token that individual instances of the client application obtain automatically on behalf of the user during the authorization process.

Ahah, you say, but the article also says:

the problem is how Twitter is using the key

Oh no! What are they possibly doing with this key that they shouldn’t be doing?

These keys are particularly significant because Twitter has configured them to enable access to special APIs which aren’t generally available yet that can be used to exchange login credentials for an access token

So in other words, with a consumer secret, you can exchange a username/password for an oAuth access token. That’s not ideal, because it means that we’re back to clients proxying usernames and passwords, but how is that a compromise? If an app successfully captures your username and password, and the app is evil, you’re screwed already! In fact, it’s probably a good thing that Twitter then pivots and says “ok, fine you got the password, but can you please just exchange it for this access token now and not store the password?” Furthermore, Twitter even says that this approach, which they call xauth, is the “least desirable way to authenticate,” and they require apps that want to use this feature to request special approval.

I don’t see a security compromise here. I see a complicated set of design issues on which Twitter is trying to make practical security decisions. It’s certainly reasonable to disagree with them, but it’s unwarranted to claim that oAuth security has been “compromised” or that Twitter made this decision “against all reason.”

spamming is a darn good explanation

The article says:

The issue here is that Twitter wants to use the keys as an abuse control mechanism to centrally disconnect spammers and other unwanted users of the service, but OAuth was simply not designed to be used for that purpose. The idea is that centrally disabling a spammer’s consumer secret key will lock out all of the spammer’s user accounts, theoretically simplifying spam control for Twitter. It’s unlikely that this naive strategy will work in practice, however.

Any spammer with a hex editor can trivially compromise the keys of popular applications and use those keys to evade Twitter’s abuse controls.

Ummm, no. As the article itself states, a spammer who takes another application’s keys still doesn’t have the access tokens, and so it cannot spam.

In fact, spamming is a very good reasons for Twitter’s use of consumer keys and secrets even in desktop clients: a spammer in this case is going to be an application that has legitimately authorized a bunch of users, and then decides to go crazy and spam their feeds. In that scenario, Twitter can very easily temporarily disable that one consumer key+secret, or throttle it, or kill it altogether, without having to trace down all the access tokens that consumer generated. I like this reason a lot.

increase the risk of phishing? No

In the past, I’ve been critical of OpenID for increasing the risk of phishing. So one might think I would agree with the article when it says:

Individual implementations aside, the general concept behind OAuth’s redirection-based authorization process materially increases the risk of phishing. The people behind the standard are fully aware of that fact, but they don’t believe that the issue should necessarily be addressed by the standard itself.

I think that’s wrong. There’s a key difference here: what’s the alternative to oAuth? The alternative is the password anti-pattern, where all those third-party apps capture your username and password. So sure, it would be good if oAuth providers had more phishing-resistant login mechanisms, say like BeamAuth (shameless plug). But, on the whole, oAuth is still a heck of a lot less phishing-prone than the password anti-pattern it’s trying to replace.

open-source and DOS: fair points

The article does make a fair point about open-source software, JavaScript-based clients, and generally the possibility that some loser might sabotage a legitimate app by copying a consumer token and secret and abusing it. If that happens a lot, then Twitter will certainly have to come up with a new approach. But let’s at least understand that there is no good approach to this problem. If no attempt is made at tracking which client is which, then one bad spamming apple could ruin it for everyone, since there’s no way to differentiate the good from the bad. Twitter may have to rethink its automatic consumer-kill policy and become a bit more nuanced, but that’s just a policy decision. At the protocol level, I don’t know how they could do much better, given that even closed-source software, as pointed out by the article, can be easily disassembled.

override the callback URL? That’s a good point

The article claims that an attacker can change the callback URL in a Twitter oAuth session, which would indeed enable a very sneaky phishing attack. I didn’t think that was the case, but it may be. I agree with the article’s take on this:

Ideally, OAuth implementors should require application developers to supply the callback address when they configure their key and should not allow that setting to be overridden by the client application in a request parameter. Twitter has a field in the key configuration that allows the developers to specify a default, but they still allow client applications to use the dangerous callback override parameter.

I think the article is right on this point. There are many reasons to disallow callback overrides, and there’s no good reason for it: oAuth web apps need cookie-based session support anyways to complete the oAuth process, there’s no need to dynamically add state to the callback URL. It would be good to do away with this “feature.”

a few very good small points

The article makes some very good smaller points that Twitter (and other oAuth providers) should heed:

• debugging oAuth is hard, and having more explicit error messages is a good idea
• logging out is confusing: if you log out of the app, are you logged out of the oAuth identity provider? Flickr does this very well with Yahoo ID integration, so oAuth and Twitter should indeed provide more guidance on this front.
• giving users more cues for trusting certain apps is probably a good idea. This can evolve over time, though, and the protocol won’t need to change.

These are all useful points. But they’re small, and they do not warrant a big scary title.

all hail oAuth 2.0!

The most confusing part of the article is the constant harping on oAuth 1.0a as “immature” when it has been deployed by dozens of high-profile providers, and the constant praise of oAuth 2.0, which has only just been released and actually weakens security to make developers’ lives easier (I’ve written before about the oAuth 2.0 problems). It’s possible that oAuth 2.0 is much better in ways I haven’t considered, but where’s the evidence? The article doesn’t say.

unnecessary insults

The article concludes:

Although I think that OAuth is salvageable and may eventually live up to the hype, my opinion of Twitter is less positive. The service seriously botched its OAuth implementation and demonstrated, yet again, that it lacks the engineering competence that is needed to reliably operate its service. Twitter should review the OAuth standard and take a close look at how Google and Facebook are using OAuth for guidance about the proper approach.

There’s no evidence of this so-called “botched” job. Sure, some smaller points are absolutely worth considering, and I applaud the author for digging deep on some of these. But I find unfortunate the overall tone of the article, the baseless and gross exaggerations, and the condescending writing that ignores reasonable justifications for Twitter’s decisions.

The voting crowd is emerging from a 2-day workshop with election officials on remote voting for military and overseas voters. I’m trying to get a sense of attendees’ impressions from that workshop, but suffice it to say that it seems to have been “exciting.” Ron Rivest compared online voting for public-office elections to drunk driving, as in “there’s no good way to do it,” and that apparently didn’t go over very well with some folks. I agree with the metaphor, however harsh it may seem.

Meanwhile, there is plenty of room for online voting in the numerous elections people hold that are not for public office: corporate boards, clubs, student government, etc. That’s why I’m excited that last night, we released Helios v3. Try it out right now by voting in our super geeky sample election and in a current-events election (Wyclef for Haiti?).

More than 25,000 votes have been cast using Helios technology. We’ve learned some very interesting lessons already, and there are many more to come. I’m hoping that, as we add more social aspects to Helios, we’ll see more usage, more data, and a unique chance to improve the technology based on real-world experience.

Browser extensions, or add-ons, can help address this issue. They can modify the behavior of specific web sites by making the browser defend user control and privacy more aggressively: they can block ads, block flash, block cookies for certain domains, add extra links for convenience (i.e. direct links to Flickr’s original resolution), etc.. Browser extensions empower users to actively defend their freedom and privacy, to push back on the more egregious actions of certain web publishers.

Except in the mobile space. Think about the iPhone browser. Apple disallows web browsers other than Safari, and there is no way to create browser extensions for Safari mobile. When you use Safari on an iPhone, you are using a browser that behaves exactly like all other iPhone Safaris, without exception. And that means that, as web publishers discover improved ways to track you, you continue to lose privacy and control over your data as you surf the Web.

This situation is getting worse: the iPad has the same limitations as the iPhone. Technically, other browsers can be installed on Android, but for all intents and purposes, it seems the built-in browser is the dominant one. Simplified computing is the norm, with single isolated applications, never applications that can modify the behavior of other applications. Thus, no browser extensions, and only one way to surf the web.

Who needs trusted computing and remote software attestation when we’re all using curated devices that, by policy and/or convenience, run only one standard, unmodified web browser?

As computing moves to the web, the Web Browser becomes the operating system, and users need ways to tweak the operating system’s behavior, or web publishers will gain far too much power. We need browser extensions. We need them to be super easy to install, like Google Chrome extensions. We need them to have extensive power to modify browser behavior, like Firefox extensions. We need them to work on mobile platforms, now.

Browser extensions used to be fun little hacks. They’ll soon be necessary to maintain a balance of power between users and publishers on the Web.

(There’s a rumor going around that Apple is about to introduce an extension framework for Safari. Whether or not this extension framework is compatible with the iPhone/iPad will be a big deal, in my opinion.)

Kim points out that two issues got confused in the flurry of press activity: the accidental collection of payload data, i.e. the URLs and web content you browsed on unsecured wifi at the moment the Google Street View car was driving by, and the intentional collection of device identifiers, i.e. the network hardware identifiers and network names of public wifi access points. Kim thinks the network identifiers are inherently more problematic than the payload, because they last for quite a bit of time, while payload data, collected for a few randomly chosen milliseconds, are quite ephemeral and unlikely to be problematic.

Kim’s right on both points. Discussion of device identifiers, which I missed in my first post, is necessary, because the data collection, in this case, was intentional, and apparently was not disclosed, as documented in EPIC’s letter to the FCC. If Google is collecting public wifi data, they should at least disclose it. In their blog post on this topic, Google does not clarify that issue.

So, Google, please tell us how long you’ve been collecting network identifiers, and how long you failed to disclose it. It may have been an oversight, but, given how much other data you’re collecting, it would really improve the public’s trust in you to be very precise here.

Now, two points:

1. taking a second look at EPIC’s letter and Kim’s original post, it still seems to me that there’s some confusion of the device identifier and payload data issues: the uproar materialized after Google revealed they had mistakenly collected payload data, and EPIC’s letter and Kim’s original post seem to weave back and forth between both issues, never really mentioning intent. Is this because the payload data story is juicier in headlines, and so bundling the two issues helps make the more important point? Maybe, but still, I think we should be more precise and careful when we attack on privacy grounds.
2. I agree that device privacy can be a big deal, especially when many people are walking around with RFIDs in their passports, pants, and with bluetooth headsets. But, in this particular case, is it a problem? If Google really only did collect the SSIDs of open, public networks that effectively invite anyone to connect to them and thus discover network name and device identifier, is that a violation of privacy, or of the Laws of Identity? I’m having trouble seeing the harm or the questionable act. Once again, these are public/open wifi networks. For the most part, these are static access points. Given Google’s stated interests in providing geolocation services, it would be detrimental to them if they catalogued roving access points. So, what’s the worst-case scenario here? Is it that, when I move to a new apartment, Google will know?

None of this excuses Google’s lack of disclosure. This was intentional data collection, it should be disclosed, period.

And it’s worth asking the questions that Kim asks, raising awareness of device privacy. I’m not sure I’m as worried as Kim is on this particular issue, but the questions are certainly legitimate.

So, in the end, the privacy advocacy theater is coming first and foremost from the EU privacy folks, who did get enraged about payload data more than anything else. There’s still some coming from EPIC and, to remain blunt, a little bit from Kim’s first post. But his second post brings up very legitimate questions, and Google should take some additional action here, at least to let us know what they were collecting, when, and whether they properly disclosed it.

I want to focus on a related problem that I’ll call privacy advocacy theater. This is a problem that my friends and colleagues are guilty of, and I’m sure I’m guilty of it at times, too. Privacy Advocacy Theater is the act of extreme criticism for an accidental data breach rather than a systemic privacy design flaw. Example: if you’re up in arms over the Google Street View privacy “fiasco” of the last few days, you’re guilty of Privacy Advocacy Theater. (If you’re generally worried about Google Street View, that’s a different problem, there are real concerns there, but I’m only talking about the collection of wifi network payload data Google performed by mistake.)

I’m looking at you, EU Privacy folks, who are investigating Google over accidental data collection. Where is your investigation of Opera, which provides Opera Mini, billed as “smarter web browsing”, smarter in the sense that it relays all data, including secure connections to your bank, through Opera’s servers? We should be much more concerned about designs that inherently create privacy risk. Oh sure, it’s easy political points to harp on accidental breaches for weeks, but it doesn’t help privacy much.

I also have to be harsh with people I respect deeply, like Kim Cameron who says that Google broke two of his very nicely crafted Laws of Identity. Come on, Kim, this was accidental data collection by code that the Google Street View folks didn’t even realize was running. (I’m giving them the benefit of the doubt. If they are lying, that’s a different problem, but no one’s claiming they’re lying, as far as I know.) The Laws of Identity apply predominantly to the systems that individuals choose to use to manage their data. If anyone is breaking the Laws of Identity, it’s the wifi access points that don’t actively nudge users towards encrypting their wifi network.

Another group I deeply admire and respect is EPIC. Here, they are also guilty of Privacy Advocacy Theater: they’re asking for an investigation into Google’s accidental wifi data collection. Now, I’m not a lawyer, and I certainly wouldn’t dare argue the law with Marc Rotenberg. But using common sense here, shouldn’t intent have something to do with this? Google did not intend to collect this data, didn’t even know they had it, and didn’t make any use of it. Shouldn’t we, instead of investigating them, help them define a process, maybe with third-party auditing from folks at EPIC, that helps them catalog what data they’re collecting, what data they’re using, etc…? At the very least, can we stop the press releases that make no distinction between intentional and unintentional data collection?

I’m getting worked up about this Privacy Advocacy Theater because, in the end, I believe it hurts privacy. Google is spending large amounts of time and money on this issue which is, as I’ve described previously, an inevitability in computer systems: accidental breaches happen all the time. We should be mostly commending them for revealing this flaw, and working with them to continue regular disclosure so that, with public oversight, these mistakes are discovered and addressed. Google has zero interest in making these mistakes. Slapping them on the wrist and having them feel some pain may be appropriate, but too much pain and too much focus on this non-issue is akin to a full-on criminal trial for driving 10 miles per hour over the speed limit: everyone’s doing it. Just fine them and move on. Then spend your time going after the folks who, by design, are endangering millions of users’ privacy.

There are plenty of real, systemic privacy issues: Facebook’s data sharing and privacy controls, Opera Mini’s design (tens of millions of users relaying all of their data to Opera, by design), Google’s intentional data retention practices, web-based ad networks, … We have enough real issues to deal with, who needs the advocacy theater?

In every instance, the companies involved didn’t mean to cause these data breaches. In every instance, they would gladly pay serious cash to prevent these bugs, given the negative publicity they cause. In every instance, most security folks I know are unfazed by these news, which they find quite unsurprising. And in each instance, the companies in question, Facebook and Google, reacted admirably: rapid disclosure, rapid response.

If you’re shocked, outraged, writing lengthy TechCrunch posts about these developments, you probably don’t understand computer security very well, and you’d better sit down with a stiff drink, because these issues are the tip of the iceberg. Accidental breaches happen all the time. Writing secure software is incredibly difficult, especially when, like Google and Facebook, you’re pushing the envelope and releasing features as quickly as possible to outpace your competitors.

We do not know how to write secure software. We do not know how to ensure that the software we write follows a given set of policies. Anyone who tries to sell you a perfectly secure system is either lying to you or doesn’t know what he’s talking about.

There are measures we can take to minimize risk, but they aren’t very sexy. Mistakes will be made, so the point is to minimize the bad consequences of those inevitable mistakes. Think of the awfully boring things you’ve heard security geeks tell you:

• don’t store user passwords in the clear.
• don’t make up your own crypto protocols, be conservative.
• defend in depth: even if you’ve got one defense, it’s always a good idea to have other defenses against the same potential breach.
• look upon large data aggregations with deep skepticism: Google, Facebook, government DNA databases, etc.
• look upon wide-ranging integration of disparate systems with deep skepticism: OpenID, Facebook Instant Personalization, etc.
• be skeptical of software vetting claims, i.e. the Apple app store.
• ask that new protocols do much more than be “no less secure” than existing protocols (I’m looking at you oAuth 2.0, no less secure than cookies).
• demand that the public be informed of all significant data breaches.

So freaking boring, right? But that’s real security. Minimize the target, minimize the risk, maximize the defenses, and always seek to strengthen security measures, because attacks only get stronger, too. And finally, provide the incentive to companies to constantly improve their defenses, by mandating disclosure.

If you’re shocked about the Facebook and Google bugs, just remember that those are the bugs we know about. There are a whole bunch more we don’t know about that Google and Facebook caught before any harm occurred. And there are a whole bunch more that nobody but bad guys know about that are actively causing harm right this minute. So the question is, as a software engineer, did you take those boring precautions to limit the damage? As a user, did you consider what data you’re storing with whom, how big a target they may be to attackers, and how motivated they are to truly secure your data?

Consider Jobs’s latest letter explaining why he won’t accept Flash on the iPhone/iPad. Most of the letter is right on: Adobe’s Flash technology on the web is slow, not open, and best replaced by HTML5. Apple has a history of ditching old technologies and pulling industry forward: they killed the floppy disk on the iMac when everyone thought it was too early, they moved to flat screens across the line before others, they embraced USB peripherals and DVI video faster than everyone else, etc. Same thing here: Flash is bloated, slow, has serious security problems, and flies in the face of the open, copy-and-paste-the-source-code Web that we know and love. Apple is leading the way in removing it from its web browser. Good for them.

And then Jobs transitions:

Adobe also wants developers to adopt Flash to create apps that run on our mobile devices. We know from painful experience that letting a third party layer of software come between the platform and the developer ultimately results in sub-standard apps and hinders the enhancement and progress of the platform.

Except this is a sleight of hand: it’s just not true. Apple already has a mechanism to control for app quality: they can reject any app for any reason. So why add this additional level of control? Why automatically reject an app that happens to be originally built using Adobe’s cross-compiler, even if that app is good? The reason is in the next paragraph:

This becomes even worse if the third party is supplying a cross platform development tool.

Again, if an app is crappy, the user simply won’t buy it. So the problem for Apple is not that cross-platform necessarily means bad app (it doesn’t), but that cross-platform means…. cross-platform. If developers can easily create apps that run on iPhone, Android, WebOS, etc. then Apple has lost a bit of its lock-in. It’s easier for you to switch to a different platform. And that is something Jobs doesn’t want.

This is by no means the first time Jobs has deftly maneuvered to maximize user lock-in while making you think it’s for your own good. Remember DRM in iTunes? This was supposedly because of the music labels insisting on DRM. We were told that, without tight control over the spread of these music files, Jobs could never have convinced the labels to move legal music online. And we were thankful that Jobs had kicked the music industry into the 21st century, and most of us were willing to swallow the bitter pill of DRM. Except, now that iTunes sells non-DRM’ed songs, Apple has maintained all sorts of limitations that appear to be gratuitous. iTunes only syncs with iPods/iPhones, and Apple went out of its way to prevent Palm’s Pre phone from connecting to iTunes. Why? Because that would make it a little bit too easy for users to stop using Apple products.

The sad thing is, I think most of Apple’s products produce sufficient lock-in thanks to quality alone. I continue to use Apple desktop and laptops, because they’re that much better. But the artificial lock-in of the iTunes/iPod/iPhone/iPad chain is beginning to make me very uncomfortable. Jobs has convinced a lot of people that this lock-in is for their own good. I don’t believe him.

UPDATE: typo fix, thanks Hacker News.

We weren’t the only folks proposing this kind of markup, and there remain healthy competing technologies. But because RDFa was architected with minimal centralization, anyone can create a vocabulary for it, anyone can use it and extend it without central approval, and that’s exactly what Yahoo, Google, and now Facebook did. They didn’t consult with the RDFa team. They didn’t have to. I consider that a great success: distributed innovation at work.

There will be work to do to reconcile the Yahoo, Google, and Facebook vocabularies. But that’s okay. RDFa lets you add as many vocabularies as you want, so you can easily combine the three vocabs for now to be maximally compatible. Over time, the tremendous power of the linked-data toolchain that forms the underpinning of RDFa will be brought to bear to progressively make the vocabularies compatible.

Exciting stuff for the structured-data Web!

Nick Carr has joined the John Gruber club, by calling us Luddites:

What these folks are ranting against, or at least gnashing their teeth over, is progress – or, more precisely, progress that goes down a path they don’t approve of. They want progress to, as Bray admits, follow their own ideological bent, and when it takes a turn they don’t like they start grumbling like granddads, yearning for the days of their idealized Apple IIs, when men were men and computers were computers.

[...]

While progress may be spurred by the hobbyist, it does not share the hobbyist’s ethic.

Back in 1999, I used to pitch open-source software to big companies. I heard very similar retorts: why would customers want to hack at the code? Let professionals handle this, this is not a job for hobbyists! Over the years, I learned the most important response to that misguided opinion: it’s not about whether hobbyists can make modifications, it’s about whether someone other than the original manufacturer can.

With open-source software, it’s not that you will necessarily hack the code yourself, but the fact that you can means that you can also hire the professional you choose to do the job for you.

Similarly, if a hybrid car comes with a sealed hood that you, a hobbyist, can’t get to even with the right tools, then you also can’t choose the professional you want to service your car. Sure, you probably want to stick with the professionals “authorized” by the auto manufacturer, especially if you’re not an expert yourself. But nothing legal or technical is going to prevent you from choosing some “unapproved professional” you trust. If you can tinker with it, then you can hire someone else to do it for you. You may be taking a risk of voiding the warranty and irreversibly breaking your hybrid car, but that’s your choice and your risk to take.

And the same thing applies to the iPad. Sure, the average consumer is probably best off using apps that have been vetted by Apple. But if the technology harshly enforces this constraint, if you can’t hack at it, then you can no longer pick the professional of your choice to hack at it, and you depend entirely on Apple. That’s bad for tinkerers and hobbyists, as I described in an earlier post, and thus it’s also bad for progress because it removes choice.

So, no, Nick, we’re not saying that every device must make every effort to be hobbyist friendly. What we’re saying is that when a company goes out of its way to prevent hobbyists from tinkering, they’re also going out of their way to prevent end-users from choosing their professional provider. That’s a departure from progress as we’ve known it throughout history. And it will mean more decisions that don’t really benefit users, like the retroactive removal of apps that are “not quite useful enough,” and similar abuses that are all too common when one company maintains such airtight control. It was bad for progress when mobile phone companies controlled their phones in this way, and it’s bad for progress when Apple does it, too.

Remember, when radios started receiving only instead of also transmitting, that was the company building a simpler device, removing features to make it simpler and cheaper. The Apple iPad is not an unprogrammable computer. It’s a programmable computer with an added feature, DRM, that actively prevents you from programming it if you’re not approved by Apple. It’s more complicated to build that way, because it’s actively blocking you from trusting another company to vet your apps.

You call that progress?

This weekend, as hundreds of thousands of people explored their iPads [...] they found [...] an application called Facebook Ultimate, featuring a sleek version of the familiar ‘f’ logo. The application quickly rose through the ranks to become one of the App Store’s top selling iPad applications. Unfortunately, it soon became apparent to these users that the application simply wasn’t very good, and that it wasn’t created by Facebook at all.

[...]

the $2.99 application rose to become the #7 top paid app on the App Store. The app received many poor reviews from upset users, plenty of which warned others that this was not the app they were looking for, but that apparently didn’t stop people from downloading it.

Tell me again how the App Store protects users?