The 7 habits of highly successful intelligence analysts

I just love these kinds of lists, that boil things down to the essentials: In a September 11 post in the Digimind blog, Orlaith Finnegan let Monica Nixon of NICS and “Bob A.”, ex Navy intelligence, put down the following 7 habits of highly successful intelligence analysts:

1) Be Organized and Disciplined
2) Communicate with Confidence, Clarity and Credibility
3) Find Meaningful Patterns in Meaningless Noise
4) Adopt a Patient, Methodical Approach
5) See the Bigger Picture
6) Be Flexible and Responsive to Change
7) Learn from Mistakes

For each point in this list, read the full description in the original article:
http://digimind.com/blog/market-industries/the-7-habits-of-highly-successful-intelligence-analysts/

Make your sources talk: elicitation, motivation, provocation… investigative journalists do it too

If you understand Swedish, you must listen to this presentation titled “The ABC of investigative journalism”, by Nils Hanson from Swedish national television (SVT). It was made during the 2012 seminar on the topic of investigative journalism held in Malmö, Sweden, during the week-end of March 23-25. This was the 16:th time Nils Hanson made this presentation.

The interesting thing here is that Nils Hanson represents the community of investigative journalists and reporters, who think of them selves as being among “the good guys”, revealing the truth to the public, uncovering what corrupt politicians hide and even sometimes shedding light on dodgy activities of government intelligence and security organizations.

However, when listening to Nils Hanson, you will hear him describe to his audience of journalists how they should go about in order to make an unwilling human source talk, how they should go about in order to make an unwilling private person agree to becoming the subject of a news story and so on.

If you have government or military intelligence training in the field of HUMINT, you will immediately notice that the methods recommended by Nils Hanson are spot-on similar to the methods used by government and military intelligence operators. The key words are elicitation, motivation, provocation, flattery, favors and favors in return and so on:

– Build trust and rapport by starting out talking about something irrelevant non-sensitive and/or slightly humouristic
– Reduce tension in a situation where the source is refusing to talk by asking for something trivial like a cigarette, and then a match and so on
– Motivate the source to talk by providing gifts without asking for anything in return and by making considerable and noticeable efforts. This will build confidence, and also a sense of indebtedness.
– When a source is refusing to be the subject of a news story or refusing to being interviewed in television, tell the source that full control is with him/her, and start moving in small steps while telling the source that he/she can back out at any time. Having committed to a recorded interview, where several people spent a lot of time, the source will seldom back out and tell them all that their efforts and work have been for nothing.

All of these methods push well-known and simple psychological buttons and leverage mechanisms of human nature such as our reluctance to jump of the band wagon once we have been on it for a while. Normal people have a strong inner voice that talks about commitment, promise, responsibility, duty, gratitude, debt, payback, fairness etc.

I am sure not many of the journalists at the Gräv 2012 seminar would feel comfortable to think of them selves as working with the same toolbox as an intelligence officer managing his human assets.

http://bambuser.com/v/2494983

Guide for Anonymous Blogging contains bad advice adding security risk

Since two years, Global Voices Online publish a guide titled “Anonymous Blogging with WordPress & Tor”:

http://advocacy.globalvoicesonline.org/projects/guide/

The guide was recently (2011-11-15) linked to from a Wired.com article by Andy Baio on the topic of reverse Google Analytics ID lookup as a method for revealing the identity of people blogging anonymously while tracking several websites with the same Google Analytics account: http://www.wired.com/epicenter/2011/11/goog-analytics-anony-bloggers/all/1

The guide from Global Voices Online, last updated in March 2009, contains a a lot of good advice and useful information, but also one serious flaw: It claims webmail services that do not provide an HTTPS connection are safe as long as you access them via TOR. This is wrong. This usage scenario adds a risk. I quote:

“Hotmail and Yahoo don’t offer secure HTTP (https) interfaces to webmail – again, this doesn’t matter so long as you use Tor every time you use these mail services. ”

In fact, the opposite is more true: Using TOR when accessing a web mail service that does not run under HTTPS – or any web based service running with regular HTTP where you submit a private username and password – poses a greater risk than not using TOR. The reason is that when using TOR, your internet traffic passes through a large number of TOR servers, all of which are set up and run by volunteers. There is no qualification mechanism for running a TOR server: anyone can set up a server, and anyone can volunteer to let their particular TOR server function as a so called Exit Node. An Exit Node is a TOR server where the TOR user’s internet traffic finally leaves the chain of TOR servers and goes out to its destination. The owner of a TOR exit node server can sniff, inspect and copy any of the data packages going out to the internet through that machine. No internet traffic from other internet users will pass through another internet user’s PC – but running a TOR server is an easy way of  “pulling in” other people’s internet traffic to pass through your machine . This opens for sniffing personal information such as email usernames and passwords – or at least email message content. Only if the webmail service is using HTTPS in all parts will the TOR exit node server be unable to read what the webmail user is receiving  and sending, since the data will be encrypted between the webmail server and the user’s browser.

So, is this big news? No. It has been well know for many years. The most famous case is from 2007, namely that of the Swedish security consultant Dan Egerstad who set up five servers, volunteering them as TOR exit node servers, and collected a massive amount of usernames and passwords, many of them from various foreign government bodies:

http://searchsecurity.techtarget.com.au/news/2240022106/Embassy-hacker-Dan-Egerstad-and-the-Tor-network

Evesdropping on TOR traffic: http://lwn.net/Articles/249388/

Researchers at Kaspersky very recently demonstrated that this kind of evesdropping goes on all the time: http://www.kaspersky.com/images/Sambuddho%20Chakravarty-10-108180.pdf

Regards, Kjell A.

Flickr picture uploads from 24 hours in print – HUGE amounts of photos

British Creative Review writes about an installation by Erik Kessels on display at Foam in Amsterdam. Kessels has printed out the amount of photos that are uploaded to Flickr during 24 hours, allegedly 1 million photos.

Creative Review writes: <<“We’re exposed to an overload of images nowadays,” says Kessels. “This glut is in large part the result of image-sharing sites like Flickr, networking sites like Facebook, and picture-based search engines. Their content mingles public and private, with the very personal being openly and un-selfconsciously displayed […] >>

What is most interesting about Kessels installation is that it turns this abstract number into something very concrete, that you can relate to physically: several rooms with piles of photos covering floor and walls.  That gets a different message though compared to the old million-billion-trillion rant.

Speaking of which, here are some more interesting figures about photos on the internet:

5 billion – Photos hosted by Flickr (September 2010).
3000+ – Photos uploaded per minute to Flickr.
130 million – At the above rate, the number of photos uploaded per month to Flickr.
3+ billion – Photos uploaded per month to Facebook.
36 billion – At the current rate, the number of photos uploaded to Facebook per year.

(Source: http://royal.pingdom.com/2011/01/12/internet-2010-in-numbers/ )

The article in Creative Review, with photos (!) showing the massive amounts of photo print-outs in the installation:

http://www.creativereview.co.uk/cr-blog/2011/november/24-hours-in-photos

This installation by Erik Kessels is on show as part of an exhibition at Foam in Amsterdam that looks at the future of photography. It features print-outs of all the images uploaded to Flickr in a 24-hour period…

Videos of presentations at DerbyCon 2011 – a must-see for anyone in information security or intelligence

During the weekend of September 30 – October 2, the DerbyCon took place in Louisville, Kentucky, at the Hyatt Regency hotel. During those three days, a number of extremely skilled and knowledgeable speakers presented on different topics in three parallel tracks. All the presentations were video recorded and are now available online.

There is a very high likelihood that you will learn valuable things from watching these videos, either from an information security standpoint, or from an open source intelligence standpoint.

http://www.irongeek.com/i.php?page=videos/derbycon1/mainlist
http://www.derbycon.com/

Use Google to search. No really.

In over 80% of all search queries made, less than five words are used.

According to the Internet monitoring company Hitwise,  the distribution of number of words used in search queries looked like this by January 2009. The statistics cover searches made, not people doing searches, which is important. Still, in over 80% of searches, less than 5 words are used in the search query. The most common search query length is 2 words.

Now, we all make a lot of searches, and in some cases we have learnt that typing one or two specific words will give us the site we are looking for on the first page of search hits. Also, a lot of people have learnt that putting a single word XYZ directly in the address field of the browser will take them to http://www.XYZ.com. Doing the same thing in Google Chrome will deliver a search on that word. People are a combination of lazy, practical, and smart – they quickly learn what works and repeats that. When they search for something for real, the more word-rich queries come in to use.

For anyone looking for something specific on the web, the chances of finding it increase if you have the knowledge to utilize the full power of the search engines. Think of it as shifting from first gear of search engine usage to second, third, fourth and fifth.

There are two dimensions to this:

1) Make sure you use the most appropriate and adequate search engine for the information you need to find. This is treated upon further in a separate article.
2) Make sure you know how to tell the search engine exactly what you are looking for, instead of throwing in a bunch of keywords in random order. This is all about making use of search operators and special characters which allows you to specify alot more complex conditions than “I want to see pages that contain these words”.

Searching more effectively with Google

There is a reason why Google is the dominating search engine: they index more, they index quicker, and they are good at understanding what results people are interested in. In fact, Google’s ability to index web content in combination with the powerful search operators specific to Google, has given birth to Google Hacking. Google hacking has nothing to do with breaching Google security. It is about using advanced searching with Google as part of the research and reconnaissance phase of a network system penetration attempt for the purpose of a) spotting targets or b) finding possible points of attack against a target.

Apart from leveraging the advanced search operators of Google in the hunt for exploit opportunities, you will of course benefit greatly in your search for information from being skilled at pushing the right buttons of the Google search engine.

Below is a list which cover what you need to know in order to make Google do a better job for you when searching. Roughly, these search operators can be put in three groups:

1) those that say what to search for, i.e. what words and numbers to match, and

2) those that say where to search, i.e. operators that limit the scope of the search or specify where the match should be,

3) those that are specialized information lookup operators, which make Google return results of a certain kind only

Operators that say WHAT to search for

1) “What” operators	Result / Effect / Meaning
secret information	Will find content that contain each of the words anywhere in the text, but not necessarily side by side
“secret information”	Will match the exact phrase and word order
~secret	Includes synonyms, alternative spellings and words with adjacent meaning
secret information OR intelligence	Will find content that contain the word secret plus either one of the words information and intelligence
intelligence -information	Will find content that contains the word intelligence while not containing the word information
intelligence +secret	Will search for content that contains intelligence and secret, with secret as required content
intelligence-community	Will find content where the two words exist separated, or written as one piece, or hyphenated
“central * agency”	The * character serves as wild card for one or more words Note! When the * character is used between two numbers in a search with no letters, it will function as a multiplication operator, returning the mathematical result multiplying the two numbers.
“US” “gov”	Google automatically includes synonyms and full-word versions of abbreviations. Putting each term in quotes assures that the search is made for exactly those terms.
“coup d’etat” 1945..1969	Will find pages that contain any number in the range 1945-1969 and the phrase “coup d’etat”

Operators that say WHERE to find a match between search term and content

2) “Where” operators	Result / Effect / Meaning
define:	Will look for the search term in word list, dictionary and glossary type of pages, e.g. define:secret
define	Alternative syntax for define:. Will look for the search term in word list, dictionary and glossary type of pages, e.g. define secret
intelligence ~glossary	Will find the word intelligence on pages that are of a glossary or dictionary or encyclopedia type
site:	Will limit the search to include only the internet domain specified, which can be a top domain, a main domain, a sub domain and so on. Examples:site:mil (combine several with the OR operator between them: site:mil OR site:gov) site:groups.google.com
inurl:	Will limit the search to only look for the search terms in the page URL. This example will show results where either one or both of wiki and sigint are part of the URL:inurl:wiki sigint
allinurl:	Very similar to inurl: but with the difference that all of the words specified must be found in the URL.
intitle:	Will limit the search to only look for the search terms in the title of pages. Title in this context means the web document HTML title, which is what you see written in the browser tab or browser window top frame.
allintitle:	Very similar to intitle: but with the difference that all of the words specified must be found in the page title.
inanchor:	Will limit the search to only look for the search terms in the anchor text of hyperlinks on pages. The anchor text is the text that was turned into a link to some page by the page creator. The anchor text may reveal something about what the page creator thinks about the page linked to, for example “Useful information on security”.
allinanchor:	Very similar to inanchor: but with the difference that all of the words specified must be found in the anchor text.
intext:	Will limit the results to include only pages where the search term was found in the text of the page.
allintext:	Very similar to intext: but with the difference that all of the words specified must be found in the text of the page.
filetype:	Will limit the results to include only files with the file extension specified, e.g. filetype:pdf to get only PDF documents
ext:	Short-hand version of filetype: that provides the exact same result
cache:	Will show the Google cache version of a web site if available, e.g. cache:cia.gov Note! This cannot be combined with additional search terms or operators
related:	Will show pages that have something in common with or are related to the site you specify, e.g. related:cia.gov Note! This cannot be combined with additional search terms or operators
link:	Will show pages that contain a link pointing to the URL you specify, e.g. link:www.cia.gov/library

Special search operators – valid only on specific Google sites

3) Special operators	Result / Effect / Meaning
location:	news.google.com – presents news search results related to the location, e.g. location:kabul
source:	news.google.com – presents news search results from the source specified, e.g. source:times
author:	groups.google.com – presents posts written by the author specified, e.g. author:einstein
group:	groups.google.com – presents posts made in the group specified, e.g. group:publicintel

When looking for information where you only have a vague idea what you should search for, only have parts of a name or only an approximate date range, advanced queries combining several such bits and pieces, involving both the OR operator, phrase quotes, and the * wild card will let you cover all bases and perform one single search that returns all possible matches.

Here are a few interesting examples that apply several of the operators listed above.

 

• PDF-files published by FBI that talk about interrogation, methods, and deception:

site:fbi.gov ext:pdf +interrogation +methods +deception

• We pages under the .mil top domain where the page title contains the word “staff”, and the page contains a link with the word “login”, excluding PDF-files as well as word documents:

site:mil intitle:staff inanchor:login -ext:pdf -ext:doc

• Excel files published with the word “internal” as part of the URL, with the phrase “internal use only” in the file:

inurl:internal ext:xls OR ext:xlsx “internal use only”

 

Learn more about how to search with Google:

http://www.googleguide.com

http://www.google.com/support/websearch/bin/answer.py?hl=en&answer=136861&rd=1

Free tools for turning search hit pages into RSS feeds

When working with environmental scanning, competitive intelligence scanning, industry monitoring, corporate reputation monitoring or any similar activity, many people use a feed reader and organize feeds on their topics and keywords of interest. A wide choice of feed readers exist, and should I mention just one, that is probably NetVibes.com. NetVibes.com is one step ahead since this (free) service allows you to organize feeds and many other types of content in a collection of tabs of your own design – all of it kept online for access and use from anywhere.

While this approach to scanning, monitoring and collecting is working fine for many people, a problem shows up when you want to monitor search results from some search engine or directory which does not provide the results as a feed of any kind.  For example, this is the case when you do a regular web seach with Google: the resuts cannot be obtained formatted as an RSS or Atom feed. So, if you are monitoring PDF documents issued by the US government or US military about piracy in the Gulf of Aden, using the following query: "Gulf of Aden" piracy ext:pdf site:(.mil OR .gov), then you cannot get those search results as a feed from Google.

The solution is to use one of a number of free services that formats any web page into a feed, making it readable and presentable by any feed reader. Raju, the owner and editor of TechPP, made a list in April 2009 of what he considers to be the top 10 services of this kind: Top 10 Free Tools to Create RSS for Any Website.

The services listed by TechPP are:

Feedity.com (not a free service)

Feed43.com

FeedYes.com

WebRSS.com

PonyFish.com

Dapper.net

Feedmarklet.com

Page2RSS.com

Feedbeater

FeedFire.com

Face recognition software is pervasive and free

http://face.com/

2010-05-03:
7 Billion Scanned Photos Later, Face.com Opens Up To Developers
http://techcrunch.com/2010/05/03/7-billion-scanned-photos-later-face-com-opens-up-to-developers/

2010-05-03:
Face.com opens its face recognition tech to devs
http://news.cnet.com/8301-27076_3-20003936-248.html

2010-06-11:
The Future of Privacy: Facial Recognition, Public Facts, and 300 Million Little Brothers
http://volokh.com/2010/06/11/the-future-of-privacy-facial-recognition-public-facts-and-300-million-little-brothers/

2010-06-16:
Police facial recognition comes to the iPhone
http://www.itworldcanada.com/news/police-facial-recognition-comes-to-the-iphone/140909

High math school grade correlated to overall high grades

On May 21, 2010, the Swedish university professor Staffan Stenhag at the University of Uppsala defended his Ph D dissertation “Betyget i matematik: Vad ger grundskolans matematikbetyg för information?”, which in english would be “Mathematics Grade: What information is provided by the school grades in mathematics?”.

The findings and conclusions presented in the dissertation are another strong argument for giving a high weighting to mathematic ability when recruiting people for intelligence work. My own personal reflection is that mathematic and logic ability is positively correlated to mental intelligence – or IQ if you wish.

The following is an abbreviated translation of an interview with Staffan Stenhag made by Susanne Sawander, published on http://www.skolporten.com/art.aspx?id=CkaUG

Pupils with high math grades also succeed in other school subjects. A possible explanation is that math studies develop the general learning ability. Stenhag got interested in the subject by observing during 25 years as a college professor how students with successful math studies would also succeed in other subjects. The dissertation starts off with the question about why pupils should study maths at junior high school level. Stenhag reviewed existing arguments. Among those arguments was the claim that mathematics serve to develop the general intellectual ability. Another argument was that mathematics serves as a selection tool when identifying the pupils most apt for higher education. In order to assess the validity of these arguments, Stenhag checked for correlations between school grades in mathematics and school grades in other subjects. His research material consisted of the grades of 124 000 Swedish pupils graduating from junior high school in 2006.  He also investigated the correlation between math grades and results in the national school exam in reading comprehension. Stenhag found that a top grade in mathematics is positively correlated to top grades overall, in other subjects as well. On the other hand, that correlation is not as strong between a top grade in Swedish, English or  social science, and the overall grades. Moreover,  Stenhag aslo found that 83 percent of pupils with a top grade in maths also got a top grade in reading comprehension. Stenhag says that it isn’t necessarily so that math studies as such result in high achievements in school. It could for example be so that the grade in math is an indicator for motivation, learning techniques, logic ability, and social conditions.  He also finds it exciting to think that studies in math might develop the general intellectual ability.

Dissertation abstract in English:

The aim of this study is to investigate what the grade in mathematics tells us about the pupil’s general academic success in other school subjects in Sweden’s compulsory school. What proficiency, except mathematical skills, does a high grade in mathematics indicate? First an inventory of the official arguments for school mathematics was conducted. The inventory shows that the arguments generally can be classified into two main classes: i) utilitarian arguments and ii) cultural arguments. In addition to these two main groups the debate also includes more remote and indirect arguments: iii) the transfer argument and iv) the selection argument .If the two last arguments are valid it is assumed that the so called indication hypothesis should be true: that pupils who succeed in mathematics also will achieve high grades in other school subjects. A statistical analysis was conducted of the grades data for the approximately 124,000 pupils who completed compulsory school in Sweden 2006. The analyses provide support for the indication hypothesis. Those pupils who manage to achieve the highest grades in mathematics often achieve high grades in other school subjects as well. This applies to both the purely theoretical and to the more practically oriented subjects. In the last phase of the study it was assumed that a possible explanation for the results could lie in the reading comprehension hypothesis; that pupils who are successful in mathematics in their ninth year of compulsory school also have good reading comprehension. This hypothesis was tested with data from the pupils’ results in the reading comprehension test that was included in the national exam in Swedish in 2006. The results provided strong support for the hypothesis. Pupils with high final grades in mathematics also have good reading comprehension. However the reverse did not apply. A good result in the reading comprehension test was not a reliable predictor of a high final grade in mathematics.

The full dissertation is available here in PDF format. It also contains an extended, 6-page abstract in english: http://www.diva-portal.org/smash/get/diva2:305754/FULLTEXT01

Learn statistics or stay stupid, misinformed and foolish

Clive Thompson of WIRED published an excellent article on April 19, 2010, on the importance of understanding probability, coincidence, correlation, causation, snap-shot samples versus trendlines, anecdotal information vs statistically valid samples, and that it is just as important as literacy.

I’m quoting some highlights form the text:

“If you don’t understand statistics, you don’t know what’s going on — and you can’t tell when you’re being lied to. Statistics should now be a core part of general education.”

“Of course, as anyone with any exposure to statistics knows, correlation is not causation. And individual stories don’t prove anything; when you examine data on the millions of vaccinated kids, even the correlation vanishes.”

“There are oodles of other examples of how our inability to grasp statistics — and the mother of it all, probability — makes us believe stupid things. Gamblers think their number is more likely to come up this time because it didn’t come up last time. Political polls are touted by the media even when their samples are laughably skewed.”

“Granted, thinking statistically is tricky. We like to construct simple cause-and-effect stories to explain the world as we experience it. “You need to train in this way of thinking. It’s not easy,” says John Allen Paulos, a Temple University mathematician.”

“That’s precisely the point. We often say, rightly, that literacy is crucial to public life: If you can’t write, you can’t think. The same is now true in math. Statistics is the new grammar.”

http://www.wired.com/magazine/2010/04/st_thompson_statistics/