The Surveillance Reforms Obama Supported Before He Was President

This story, written by Jonathan Stray for ProPublica, was originally published on Aug. 7, 2013.

When the House of Representatives recently considered an amendment that would have dismantled the NSA’s bulk phone records collection program, the White House swiftly condemned the measure. But only five years ago, Sen. Barack Obama, D-Ill. was part of a group of legislators that supported substantial changes to NSA surveillance programs. Here are some of the proposals the president co-sponsored as a senator.

As a senator, Obama wanted to limit bulk records collection.

Obama co-sponsored a 2007 bill, introduced by Sen. Russ Feingold, D-Wis., that would have required the government to demonstrate, with “specific and articulable facts,” that it wanted records related to “a suspected agent of a foreign power” or the records of people with one degree of separation from a suspect. The bill died in committee. Following pressure from the Bush administration, lawmakers had abandoned a similar 2005 measure, which Obama also supported.

We now know the Obama administration has sought, and obtained, the phone records belonging to all Verizon Business Network Services subscribers (and reportedly, Sprint and AT&T subscribers, as well). Once the NSA has the database, analysts search through the phone records and look at people with two or three degrees of separation from suspected terrorists.

The measure Obama supported in 2007 is actually similar to the House amendment that the White House condemned earlier this month. That measure, introduced by Reps. Justin Amash, R-Mich., and John Conyers, D-Mich., would have ended bulk phone records collection but still allowed the NSA to collect records related to individual suspects without a warrant based on probable cause.

The 2007 measure is also similar to current proposals introduced by Conyers and Sen. Bernie Sanders, I-Vt.

As a senator, Obama wanted to require government analysts to get court approval before accessing incidentally collected American data.

In Feb. 2008, Obama co-sponsored an amendment, also introduced by Feingold, which would have further limited the ability of the government to collect any communications to or from people residing in the U.S.  

The measure would have also required government analysts to segregate all incidentally collected American communications. If analysts wanted to access those communications, they would have needed to apply for individualized surveillance court approval.

The amendment failed 35-63. Obama later reversed his position and supported what became the law now known to authorize the PRISM program. That legislation 2014 the FISA Amendments Act of 2008 2014 also granted immunity to telecoms that had cooperated with the government on surveillance.

The law ensured the government would not need a court order to collect data from foreigners residing outside the United States. According to the Washington Post, analysts are told that they can compel companies to turn over communications if they are 51 percent certain the data belongs to foreigners.

Powerpoint presentation slides published by the Guardian indicate that when analysts use XKeyscore 2014 the software the NSA uses to sift through huge amounts of raw internet data 2014 they must first justify why they have reason to believe communications are foreign. Analysts can select from rationales available in dropdown menus and then read the communications without court or supervisor approval.

Finally, analysts do not need court approval to look at previously-collected bulk metadata either, even domestic metadata. Instead, the NSA limits access to incidentally collected American data according to its own “minimization” procedures. A leaked 2009 document said that analysts only needed permission from their “shift coordinators” to access previously-collected phone records. Rep. Stephen Lynch, D-Mass., has introduced a bill that would require analysts to get special court approval to search through telephone metadata.

As a senator, Obama wanted the executive branch to report to Congress how many American communications had been swept up during surveillance.

Feingold’s 2008 amendment, which Obama supported, would have also required the Defense Department and Justice Department to complete a joint audit of all incidentally collected American communications and provide the report to congressional intelligence committees. The amendment failed 35-63.

The Inspector General of the Intelligence Community told Senators Ron Wyden, D-Ore., and Mark Udall, D-Co. last year that it would be unfeasible to estimate how many American communications have been incidentally collected, and doing so would violate Americans’ privacy rights.

As a senator, Obama wanted to restrict the use of gag orders related to surveillance court orders.

Obama co-sponsored at least two measures that would have made it harder for the government to issue nondisclosure orders to businesses when compelling them to turn over customer data.

One 2007 bill would have required the government to demonstrate that disclosure could cause one of six specific harms: by either endangering someone, causing someone to avoid prosecution, encouraging the destruction of evidence, intimidating potential witnesses, interfering with diplomatic relations, or threatening national security. It would have also required the government to show that the gag order was “narrowly tailored” to address those specific dangers. Obama also supported a similar measure in 2005. Neither measure made it out of committee.

The Obama administration has thus far prevented companies from disclosing information about surveillance requests. Verizon’s surveillance court order included a gag order.

Meanwhile, Microsoft and Google have filed motions with the Foreign Intelligence Surveillance Court seeking permission to release aggregate data about directives they’ve received. Microsoft has said the Justice Department and the FBI had previously denied its requests to release more information. The Justice Department has asked for more time to consider lifting the gag orders.

As a senator, Obama wanted to give the accused a chance to challenge government surveillance.

Obama co-sponsored a 2007 measure that would have required the government to tell defendants before it used any evidence collected under the controversial section of the Patriot Act. (That section, known as 215, has served as the basis for the bulk phone records collection program.) Obama also supported an identical measure in 2005.

Both bills would have ensured that defendants had a chance to challenge the legalityof Patriot Act surveillance. The Supreme Court has since held that plaintiffs who cannot prove they have been monitored cannot challenge NSA surveillance programs.

Those particular bills did not make it out of committee. But another section of the Foreign Intelligence Surveillance Act requires that the government tell defendants before it uses evidence collected under that law.

Until recently, federal prosecutors would not tell defendants what kind of surveillance had been used.

The New York Times reported that in two separate bomb plot prosecutions, the government resisted efforts to reveal whether its surveillance relied on a traditional FISA order, or the 2008 law now known to authorize PRISM. As a result, defense attorneys had been unable to contest the legality of the surveillance. Sen. Dianne Feinstein, D-Calif., later said that in both cases, the government had relied on the 2008 law, though prosecutors now dispute that account.

On July 30, the Justice Department reversed its position in one bomb plot prosecution. The government disclosed that it had not gathered any evidence under the 2008 law now known to authorize sweeping surveillance.

But that’s not the only case in which the government has refused to detail its surveillance. When San Diego cab driver BasaalySaeedMoalin was charged with providing material support to terrorists based on surveillance evidence in Dec. 2010, his attorney, Joshua Dratel, tried to get the government’s wiretap application to the Foreign Intelligence Surveillance Court. The government refused, citing national security.

Dratel only learned that the government had used Moalin’s phone records as the basis for its wiretap application 2014 collected under Section 215 of the Patriot Act 2014 when FBI Deputy Director Sean Joyce cited the Moalin case as a success story for the bulk phone records collection program.

Reuters has also reported that a U.S. Drug Enforcement Administration unit uses evidence from surveillance to investigate Americans for drug-related crimes, and then directs DEA agents to “recreate” the investigations to cover up the original tip, so defendants won’t know they’ve been monitored.

As a senator, Obama wanted the attorney general to submit a public report giving aggregate data about how many people had been targeted for searches.

Under current law, the attorney general gives congressional intelligence committees a semiannual report with aggregate data on how many people have been targeted for surveillance. Obama co-sponsored a 2005 bill that would have made that report public. The bill didn’t make it out of committee.

Despite requests from Microsoft and Google, the Justice Department has not yet given companies approval to disclose aggregate data about surveillance directives.

As a senator, Obama wanted the government to declassify significant surveillance court opinions.

Currently, the attorney general also gives congressional intelligence committees “significant” surveillance court opinions, decisions and orders and summaries of any significant legal interpretations. The 2005 bill that Obama co-sponsored would have released those opinions to the public, allowing redactions for sensitive national security information.

Before Edward Snowden’s disclosures, the Obama Justice Department had fought Freedom of Information Act lawsuits seeking surveillance court opinions. On July 31, the Director of National Intelligence released a heavily redacted version of the FISA court’s “primary order” compelling telecoms to turn over metadata.

In response to a request from Yahoo, the government also says it is going to declassify court documents showing how Yahoo challenged a government directive to turn over user data. The Director of National Intelligence is still reviewing if there are other surveillance court opinions and other significant documents that may be released. Meanwhile, there are several bills in Congress that would compel the government to release secret surveillance court opinions.

FAQ: What You Need to Know About the NSA’s Surveillance Programs

This story, written by Jonathan Stray for ProPublica, was originally published on June 27, 2013.

There have been a lot of news stories about NSA surveillance programs following the leaks of secret documents by Edward Snowden. But it seems the more we read, the less clear things are. We’ve put together a detailed snapshot of what’s known and what’s been reported where.

What information does the NSA collect and how?

We don’t know all of the different types of information the NSA collects, but several secret collection programs have been revealed:

A record of most calls made in the U.S., including the telephone number of the phones making and receiving the call, and how long the call lasted. This information is known as “metadata” and doesn’t include a recording of the actual call (but see below). This program was revealed through a leaked secret court order instructing Verizon to turn over all such information on a daily basis. Other phone companies, including AT&T and Sprint, also reportedly give their records to the NSA on a continual basis. All together, this is several billion calls per day.

Email, Facebook posts and instant messages for an unknown number of people, via PRISM, which involves the cooperation of at least nine different technology companies. Google, Facebook, Yahoo and others have denied that the NSA has “direct access” to their servers, saying they only release user information in response to a court order. Facebook has revealed that, in the last six months of 2012, they handed over the private data of between 18,000 and 19,000 users to law enforcement of all types — including local police and federal agencies, such as the FBI, Federal Marshals and the NSA.

Massive amounts of raw Internet traffic The NSA intercepts huge amounts of raw data, and stores billions of communication records per day in its databases. Using the NSA’s XKEYSCORE software, analysts can see “nearly everything a user does on the Internet” including emails, social media posts, web sites you visit, addresses typed into Google Maps, files sent, and more. Currently the NSA is only authorized to intercept Internet communications with at least one end outside the U.S., though the domestic collection program used to be broader. But because there is no fully reliable automatic way to separate domestic from international communications, this program also captures some amount of U.S. citizens’ purely domestic Internet activity, such as emails, social media posts, instant messages, the sites you visit and online purchases you make.

The contents of an unknown number of phone calls There have been several reports that the NSA records the audio contents of some phone calls and a leaked document confirms this. This reportedly happens “on a much smaller scale” than the programs above, after analysts select specific people as “targets.” Calls to or from U.S. phone numbers can be recorded, as long as the other end is outside the U.S. or one of the callers is involved in “international terrorism“. There does not seem to be any public information about the collection of text messages, which would be much more practical to collect in bulk because of their smaller size.

The NSA has been prohibited from recording domestic communications since the passage of the Foreign Intelligence Surveillance Act but at least two of these programs — phone records collection and Internet cable taps — involve huge volumes of Americans’ data.

Does the NSA record everything about everyone, all the time?

The NSA records as much information as it can, subject to technical limitations (there’s a lot of data) and legal constraints. This currently includes the metadata for nearly all telephone calls made in the U.S. (but not their content) and massive amounts of Internet traffic with at least one end outside the U.S. It’s not clear exactly how many cables have been tapped, though we know of at least one inside the U.S., a secret report about the program by the NSA’s Inspector General mentions multiple cables, and the volume of intercepted information is so large that it was processed at 150 sites around the world as of 2008. We also know that Britain’s GCHQ, which shares some intelligence with the NSA, had tapped over 200 cables as of 2012, belonging to seven different telecommunications companies.

Until 2011 the NSA also operated a domestic Internet metadata program which collected mass records of who emailed who even if both parties were inside the U.S.

Because it is not always possible to separate domestic from foreign communications by automatic means, the NSA still captures some amount of purely domestic information, and it is allowed to do so by the Foreign Intelligence Surveillance Court.

The collected information covers “nearly everything a user does on the Internet,” according to a presentation on the XKEYSCORE system. The slides specifically mention emails, Facebook chats, websites visited, Google Maps searches, transmitted files, photographs, and documents of different kinds. It’s also possible to search for people based on where they are connecting from, the language they use, or their use of privacy technologies such as VPNs and encryption, according to the slides.

This is a massive amount of data. The full contents of intercepted Internet traffic can only be stored for up to a few days, depending on the collection site, while the associated “metadata” (who communicated with whom online) is stored up to 30 days. Telephone metadata is smaller and is stored for five years. NSA analysts can move specific data to more permanent databases when they become relevant to an investigation.

The NSA also collects narrower and more detailed information on specific people, such as the actual audio of phone calls and the entire content of email accounts. NSA analysts can submit a request to obtain these types of more detailed information about specific people.

Watching a specific person like this is called “targeting” by the Foreign Intelligence Surveillance Act, the law which authorizes this type of individual surveillance. The NSA is allowed to record the conversations of non-Americans without a specific warrant for each person monitored, if at least one end of the conversation is outside of the U.S. It is also allowed to record the communications of Americans if they are outside the U.S. and the NSA first gets a warrant for each case. It’s not known exactly how many people the NSA is currently targeting, but according to a leaked report the NSA intercepted content from 37,664 telephone numbers and email addresses from October 2001 to January 2007. Of these, 8% were domestic: 2,612 U.S. phone numbers and 406 U.S. email addresses.

How the NSA actually gets the data depends on the type of information requested. If the analyst wants someone’s private emails or social media posts, the NSA must request that specific data from companies such as Google and Facebook. Some technology companies (we don’t know which ones) have FBI monitoring equipment installed “on the premises” and the NSA gets the information via the FBI’s Data Intercept Technology Unit. The NSA also has the capability to monitor calls made over the Internet (such as Skype calls) and instant messaging chats as they happen.

For information that is already flowing through Internet cables that the NSA is monitoring, or the audio of phone calls, a targeting request instructs automatic systems to watch for the communications of a specific person and save them.

It’s important to note that the NSA probably has information about you even if you aren’t on this target list. If you have previously communicated with someone who has been targeted, then the NSA already has the content of any emails, instant messages, phone calls, etc. you exchanged with the targeted person. Also, your data is likely in bulk records such as phone metadata and Internet traffic recordings. This is what makes these programs “mass surveillance,” as opposed to traditional wiretaps, which are authorized by individual, specific court orders.

What does phone call metadata information reveal, if it doesn’t include the content of the calls?

Even without the content of all your conversations and text messages, so-called “metadata” can reveal a tremendous amount about you. If they have your metadata, the NSA would have a record of your entire address book, or at least every person you’ve called in the last several years. They can guess who you are close to by how often you call someone, and when. By correlating the information from multiple people, they can do sophisticated “network analysis” of communities of many different kinds, personal or professional — or criminal.

Phone company call records reveal where you were at the time that a call was made, because they include the identifier of the radio tower that transmitted the call to you. The government has repeatedly denied that it collects this information, but former NSA employee Thomas Drake said they do. For a sense of just how powerful location data can be, see this visualization following a German politician everywhere he goes for months, based on his cellphone’s location information.

Even without location data, records of who communicated with whom can be used to discover the structure of groups planning terrorism. Starting from a known “target” (see above), analysts typically reconstruct the social network “two or three hops” out, examining all friends-of-friends, or even friends-of-friends-of-friends, in the search for new targets. This means potentially thousands or millions of people might be examined when investigating a single target.

Metadata is a sensitive topic because there is great potential for abuse. While no one has claimed the NSA is doing this, it would be possible to use metadata to algorithmically identify, with some accuracy, members of other types of groups like the Tea Party or Occupy Wall Street, gun owners, undocumented immigrants, etc. An expert in network analysis could start with all of the calls made from the time and place of a protest, and trace the networks of associations out from there.

Phone metadata is also not “anonymous” in any real sense. The NSA already maintains a database of the phone numbers of all Americans for use in determining whether someone is a “U.S. person” (see below), and there are several commercial number-to-name services in any case. Phone records become even more powerful when they are correlated with other types of data, such as social media posts, local police records and credit card purchase information, a process known as intelligence fusion.

Does the NSA need an individualized warrant to listen to my calls or look at my emails?

It’s complicated, but not in all cases. Leaked court orders set out the “minimization” procedures that govern what the NSA can do with the domestic information it has intercepted. The NSA is allowed to store this domestic information because of the technical difficulties in separating foreign from domestic communications when large amounts of data are being captured.

Another document shows that individual intelligence analysts make the decision to look at previously collected bulk information. They must document their request, but only need approval from their “shift coordinator.” If the analyst later discovers that they are looking at the communications of a U.S. person, they must destroy the data.

However, if the intercepted information is “reasonably believed to contain evidence of a crime” then the NSA is allowed to turn it over to federal law enforcement. Unless there are other (still secret) restrictions on how the NSA can use this data this means the police might end up with your private communications without ever having to get approval from a judge, effectively circumventing the whole notion of probable cause.

This is significant because thousands or millions of people might fall into the extended social network of a single known target, but it is not always possible to determine whether someone is a U.S. person before looking at their data. For example, it’s not usually possible to tell just from someone’s email address, which is why the NSA maintains a database of known U.S. email addresses and phone numbers. Internal documents state that analysts need only “51% confidence” that someone is a non-U.S. person before looking at their data, and if the NSA does not have “specific information” about someone, that person is “presumed to be a non-United States person.”

Also, the NSA is allowed to provide any of its recorded information to the FBI, if the FBI specifically asks for it.

Is all of this legal?

Yes, assuming the NSA adheres to the restrictions set out in recently leaked court orders. By definition, the Foreign Intelligence Surveillance Court decides what it is legal for the NSA to do. But this level of domestic surveillance wasn’t always legal, and the NSA’s domestic surveillance program has been found to violate legal standards on more than one occasion.

The NSA was gradually granted the authority to collect domestic information on a massive scale through a series of legislative changes and court decisions over the decade following September 11, 2001. See this timeline of loosening laws. The Director of National Intelligence says that authority for PRISM programs comes from section 702 of the Foreign Intelligence Surveillance Act and the Verizon metadata collection order cites section 215 of the Patriot Act. The author of the Patriot Act disagrees that the act justifies the Verizon metadata collection program.

The NSA’s broad data collection programs were originally authorized by President Bush on October 4, 2001. The program operated that way for several years, but in March 2004 a Justice Department review declared the bulk Internet metadata program was illegal. President Bush signed an order re-authorizing it anyway. In response, several top Justice Department officials threatened to resign, including acting Attorney General James Comey and FBI director Robert Mueller. Bush backed down, and the Internet metadata program was suspended for several months. By 2007, all aspects of the program were re-authorized by court orders from the Foreign Intelligence Surveillance Court.

In 2009, the Justice Department acknowledged that the NSA had collected emails and phone calls of Americans in a way that exceeded legal limitations.

In October 2011, the Foreign Intelligence Surveillance Court ruled that the NSA violated the Fourth Amendment at least once. The Justice Department has said that this ruling must remain secret, but we know it concerned some aspect of the “minimization” rules the govern what the NSA can do with domestic communications. The Foreign Intelligence Surveillance Court recently decided that this ruling can be released, but Justice Department has not yet done so.

Civil liberties groups including the EFF and the ACLU dispute the constitutionality of these programs and have filed lawsuits to challenge them.

How long can the NSA keep information on Americans?

The NSA can generally keep intercepted domestic communications for up to five years. It can keep them indefinitely under certain circumstances, such as when the communication contains evidence of a crime or when it’s “foreign intelligence information,” a broad legal term that includes anything relevant to “the conduct of the foreign affairs of the United States.”

The NSA can also keep encrypted communications indefinitely. That includes any information sent to or from a secure web site, that is, a site with a URL starting with “https”.

Does the NSA do anything to protect Americans’ privacy?

Yes. First, the NSA is only allowed to intercept communications if at least one end of the conversation is outside of the U.S. — though it doesn’t have to distinguish domestic from foreign communication until the “earliest practicable point” which allows the NSA to record bulk information from Internet cables and sort it out later. When the NSA discovers that previously intercepted information belongs to an American, it must usually destroy that information. Because this determination cannot always be made by computer, this sometimes happens only after a human analyst has already looked at it.

The NSA also must apply certain safeguards. For example, the NSA must withhold the names of U.S. persons who are not relevant to ongoing investigations when they distribute information — unless that person’s communications contain evidence of a crime or are relevant to a range of national security and foreign intelligence concerns.

Also, analysts must document why they believe someone is outside of the U.S. when they ask for addition information to be collected on that person. An unknown number of these cases are audited internally. If the NSA makes a mistake and discovers that it has targeted someone inside the U.S., it has five days to submit a report to the Department of Justice and other authorities.

What if I’m not an American?

All bets are off. There do not appear to be any legal restrictions on what the NSA can do with the communications of non-U.S. persons. Since a substantial fraction of the world’s Internet data passes through the United States, or its allies, the U.S. has the ability to observe and record the communications of much of the world’s population. The European Union has already complained to the U.S. Attorney General.

The U.S. is hardly the only country doing mass surveillance, though its program is very large. GCHQ, which is the British counterpart to the NSA, has a similar surveillance program and shares data with the NSA. Many countries now have some sort of mass Internet surveillance now in place. Although passive surveillance is often hard to detect, more aggressive governments use intercepted information to intimidate or control their citizens, including Syria, Iran, Egypt, Bahrain and China. Much of the required equipment is sold to these governments by American companies.