Make us your home page
Instagram

Today’s top headlines delivered to you daily.

(View our Privacy Policy)

How the NSA filters an overload of data

The National Security Agency's Special Source Operations branch manages "partnerships" in which U.S. and foreign telecommunications companies allow the NSA to use their facilities to intercept phone calls, emails and other data. This briefing describes problems with overcollection of data from e-mail address books and buddy lists, as well as NSA efforts to filter out what it does not need.

- - - - -

What is a "session"?

A session is another term for a data interchange between two computers, such as when you log into a service or mail is transfered. Each of these "sessions" crosses the NSA's collection points, filling storage repositories with redundant data.

"Selectors detasked"

Selectors are the NSA's term for what it is searching for - such as an email address or phone number. Detasking means the agency stops collection. One slide laments that the Yahoo Messenger problem forced it to stop collecting important information about Greece and Libya.

How many address books are collected?

This slide sets out the number of contact lists collected on a single day, Jan. 10, 2012, from the six top overseas access points, which are designated by alphanumeric codes. The "US" prefix denotes an NSA access point and "DS" refers to the NSA's Australian counterpart.

MARINA/MAINWAY/PINWALE

MARINA is an NSA database and analysis tool for internet metadata. MAINWAY is primarily for telephone metadata for contact chaining, and PINWALE for written content.

"Attributable"

Address books make up an unexpectedly large share of information pulled in by the NSA. Many of them are less useful to the NSA because they are "unattributed," with the owners unknown.

Why collect "buddy lists"?

Buddy lists sometimes include the text of messages waiting to be delivered, which count as content. Webmail inboxes, which list new messages, often include a line or two of the text.

"500,000 buddy lists and inboxes collected on a representative day"

When the NSA searches for a specific target, such as an email address used by a terrorist, it usually finds only a listing in someone else's address book. More valuable finds - the target's own address book, a person communicating with the target or a message that mentions the target - are rarer.

A targeted account gets hacked

Four slides tell the story of a Yahoo email account, under NSA surveillance, that was hacked and subsquentaly used by spammers to send bulk mail. S2E is the Middle East and North Africa office of the NSA's Analysis and Production subdirectorate. The user of this email account had a number of Yahoo groups in his or her address book, some of them with thousands of members. Spammers used the account to send emails to all of them. The spam created so many false connections that the Yahoo account had to be "emergency detasked" to prevent the collection system from overflowing.

- - - - -

This is a glance at problems with the National Security Agency's overcollection of address books and buddy lists and its efforts to weed out useless content.

SCISSORS

SCISSORS is an NSA system that helps parse electronic communications. There are five kinds of data, details unknown, that are collected at the four named access points.

Ownerless address books blocked by SCISSORS

For "ownerless," address books, which the NSA cannot attribute to a specific account holder, SCISSORS tries to block collection of content. (Graphic includes chart that shows how often that happened in mid-summer 2012.)

Ownerless address books blocked, by points of access

This chart displays the same data as the previous slide, separated by "signals intelligence address," or point of access.

Emergency detasks

Improved filtering between late 2011 and mid-2012 allowed the NSA to reduce the number of accounts for which it had to stop collection urgently.

SIGDEV

SIGDEV is signals intelligence development, or analysis of data flows to discover new forms of useful information.

"Shifting collection philosophy at NSA"

Accustomed to siphoning in as much electronic data as possible, in case it proves useful later, the NSA (according to the authors of this presentation) needs to become more selective. One slide's bullet point: The "shifting collection philosophy at NSA is "Memorialize what you need" versus "Order one of everything off the menu and eat what you want."

- - - - -

This excerpt is from an article in the NSA's "Intellipedia," a classified system built with the same open-source software used by Wikipedia. Like the other documents, it describes the problem of high-volume, low-value data collection - and the NSA's response - with a focus on Internet contact lists.

How SCISSORS blocks collection

Most address books are targeted simply because they are address books, without specifying a foreign target. Using the SCISSORS tool, NSA will try to prevent useless content from being sent to the PINWALE repository. Even for "ownerless" address books, however, the NSA keeps the metadata that links each of the contacts.

XKEYSCORE

The NSA is trying to filter out unwanted data at the point of collection, using a selection tool called XKEYSCORE, rather than send everything to central repositories for processing by SCISSORS.

"Little or no useful FI information"

SCISSORS was necessary because "unattributed address books" account for collection of large amout of data with "little or no 'foreign intelligence' information."

How the NSA filters an overload of data 10/15/13 [Last modified: Tuesday, October 15, 2013 11:41am]
Photo reprints | Article reprints

© 2017 Tampa Bay Times

    

Join the discussion: Click to view comments, add yours

Loading...
  1. Rowdies shut out at Pittsburgh

    Soccer

    PITTSBURGH — The Rowdies lost their first USL game in nearly a month, 1-0 to Pittsburgh on Thursday night.

  2. Trump reveals that he didn't record Comey after all

    Politics

    WASHINGTON — President Donald Trump declared Thursday he never made and doesn't have recordings of his private conversations with ousted former FBI director James Comey, ending a monthlong guessing game that he started with a cryptic tweet and that ensnared his administration in yet more controversy.

    President Donald Trump said Thursday that he didn’t record his conversations with James Comey.
  3. Lightning fans, don't get attached to your first-round draft picks

    Lightning Strikes

    CHICAGO — When Lightning GM Steve Yzerman announces his first-round pick tonight in the amateur draft at No. 14, he'll invite the prospect onto the stage for the once-in-a-lifetime photo opportunity.

    Tampa Bay Lightning left wing Jonathan Drouin (27) eludes  Montreal Canadiens left wing Phillip Danault (24) during the second period of Wednesday???‚??„?s (12/28/16) game between the Tampa Bay Lightning and the Montreal Canadiens at the Amalie Arena in Tampa.
  4. Investigation Discovery TV show profiles 2011 Landy Martinez murder case

    Crime

    The murder of a St. Petersburg man will be featured this week on a new true crime series Murder Calls on Investigation Discovery.

    Jose Adame sits in a Pinellas County courtroom during his 2016 trial and conviction for first-degree murder. Adame was convicted of first-degree murder last year for torturing and then executing his boyfriend as he pleaded for his life in 2011. Now it will be featured in a new true crime series Murder Calls on Investigation Discovery. The episode will air on June 26 at 9 p.m. [DOUGLAS R. CLIFFORD   |   Times]
  5. Uhuru mayoral candidate Jesse Nevel protests exclusion from debate

    Blogs

    ST. PETERSBURG — Jesse Nevel, the International People’s Democratic Uhuru Movement candidate for mayor, on Thursday demanded that he be allowed to participate in a July 25 televised debate between incumbent Mayor Rick Kriseman and challenger Rick Baker.

    Mayoral candidate Jesse Nevel holds a news conference outside the headquarters of the Tampa Bay Times on Thursday to protest his exclusion from the mayoral debate. Nevel is a member of the International People’s Democratic Uhuru Movement.