Below are translations of “Open Source Intelligence” into a number of languages:
French: Renseignement de source ouverte
German: Offene Informationsgewinnung
Italian: Informazioni di fonti aperte
Norwegian: Etterretning fra åpne kilder
Russian: Разведка по открытым источникам
Spanish: Inteligencia de fuente abierta
Swedish: Underrättelser från öppna källor
The International Relations and Security Network (ISN) is the world’s leading open access information service for international relations and security professionals. In a 12-minute podcast from October 12, 2009, professor Arthur S Hulnick is interviewed. He provides a brief history of the use of OSINT, and touches upon it’s limitations in relation to HUMINT. He also underlines the importance of source criticism and information evaluation.
A picture illustrating the intelligence cycle model
The so called Intelligence Cycle is a model used world-wide for the process of producing intelligence. The model is a feedback-loop where results, findings and conclusions re-shape the direction and prioritizations of the intelligence production. The parts of the intelligence cycle are therefore often depicted as sectors of a circle, with arrows pointing from one step of the cycle to the next. The intelligence cycle is of course a theoretic construction which can be challenging to fully comply with in practice. Nonetheless, it is a very useful model to follow when structuring and organizing any kind intelligence production work. The model is valid as a structure of work for all from one-man operations to giant operations with thousands of people involved.
These are the parts or steps in the intelligence cycle. In all large organizations, with enough staff resources, all these phases are going on in parallel.
Also referred to as: Direction and Planning
This is done in this phase: Definition of intelligence needs and requirements, formulation of questions you want to answer, prioritization of time and resources, selection of collection methods, specification of sources and more. Within NATO, a standard term used instead of “intelligence requirements” is EEI, Essential Elements of Intelligence.
This is done in this phase: Based on the intelligence needs and requirements, one or several types of intelligence may be collected. Such types are for example OSINT, HUMINT, IMINT, SIGINT, ELINT, COMINT, MASINT. Regarding OSINT, different types of sources that can be involved are listed in this article: OSINT Source Typology
This is done in this phase: Collected material is organized – collated – in some form, which could be all from putting xeroxed articles in a binder, to entering material in a digital database along with various descriptive data etc. Texts may need translation, or classification in terms of type, origin, subject, producer and more. A reliability rating can be assigned to the material. Raw data may need to be recorded in a structured format, along with metadata which turns it into information.
Also referred to as: Exploitation.
This is done in this phase: Analysis of the collected material is about interpreting the information and conclude what it means in relation to the intelligence requirements. Bits and pieces from different sources are compared and synthesised in order to create a bigger picture where things such as chains of events or event patterns or correlations can be confirmed, or motivate a hypothesis formulation which can be tested through further collection and analysis. The questions that have been formulated as part of the intelligence requirements should be answered.
Also referred to as: Presentation, Delivery
This is done in this phase: The intelligence produced in the analysis phase is packaged, presented and delivered to the decision maker, client, reciever, superior officer etc, in some shape or form. Dissemination can be done in many forms – traditionally, a regular, written report on paper is a common format. Lately, digital media has become an increasingly preferred packaging of intelligence products.
This is done in this phase: Based on insights from disseminated intelligence as well as on new events, conditions and preferences, the client (decision maker, reciever, superior officer) provides feedback which is taken in to account in the Direction phase.
General daily newspapers
Industry newspapers
Industry journals
Professional journals
Company internal journals
Research papers
Dictionaries and reference literature
Non-fiction books in general
Annual reports from companies and other organizations
Government public documents
Employment Advertisements
Television news
Documentary films
Other non-entertainment shows
Radio news
Radio documentaries
Other non-entertainment radio shows
Online news papers
Online journals
Blogs and microblogs (e.g. Blogger, WordPress, Twitter)
Social communities (e.g. Facebook, LinkedIn, MySpace, Orkut)
Discussion forums
Newsgroups (Usenet Newsgroups)
IRC (Internet Relay Chat)
Wikis (e.g. Wikipedia, Wikisummaries, Wikileaks)
Sharing services for video (YouTube is dominating, but there are several others listed here and compared here)
Sharing services for photos (e.g. Panoramio, Flickr, and a number of others listed here)
NGO websites
Company websites
Governmental websites
Government public documents
Email newsletters
Employment advertisements
Published patents and patent applications
Domain registrar WHOIS records
Audio podcasts
Video podcasts
Maps
Satellite imagery
Aerial photos
Metadata extracted from published content (in practice from image files, office suite files and PDFs)
Public statistics databases
Newspaper articles databases (for material from before the Internet)
Business databases (for facts on companies)
Library & bookshop databases, and books.google.com
Documentary film productions databases (e.g. IMDb.com)
White pages phone books
Yellow pages phone books
People seach engines (e.g. Pipl.com)
All kinds of specialty search engines and directories, the content of which is often referred to as the “Deep Web”
Domain registrars WHOIS records
This is a living document which will be edited, updated and expanded over time.
Below are some definitions of concepts that are frequenty used on iOSINT:
To start with, there are important differences between Data, Information, and Intelligence.
Data are letters and numbers without context – words and numbers that does not have a meaning by themselves. For example, the following numbers are meaningless and useless if we don’t know what they describe: “184, 286, 172, 372”
Metadata are data about data – they describe what some other data is. While an example of data is “12345”, an example of metadata related to that data could be “zipcode”.
Information is data with a context – words and numbers that have a meaning because we have enough metadata to let us understand how the data should be interpreted and turned into information, or because there is enough structure or context to let us make sense of the data. An example of how structure or context will let us make use of data by interpreting the data correctly and turn it into information is this: “93650 Fresno, 90745 Long Beach, 90080 Los Angeles”. From previous knowledge, we know that a five-digit number placed next to the name of a town or city is a postal code or zip code.
In lack of structure or context, time can also help making sense of data and turning it into information. For example, without additional metadata or context, we can conclude that a long row of five-digit number are zipcodes provided that we have enough time to compare and check the numbers.
Swedish professor in computer science Börje Langefors formulated the so called Information Equation which describes this relationship:
I=f(d,s,t)
which should be read “Information is a function of data, structure, and time”.
Intelligence is more than information. Intelligence is qualified information – sometimes described as “information for action”. A definition of Intelligence which I think captures the essential meaning is “a product of evaluated information from a wide range of sources that supports a decision making process”. (In the context of OSINT, intelligence is of course a concept different from the psychological term intelligence, meaning our mental capacity to understand, think and reason).
Open source when speaking about information and intelligence means publicly available and non-exclusive sources of data and information that anyone can access and use. Open sources include all kinds of media such as papers, journals, radio and television, public statistics, public directories, published scientific research and more. The internet is a hugh publication channel for additional open source information, which otherwise would have been hard to find: amateur photographs, satellite imagery, individuals’ experiences from products and places etc etc. (In the context of OSINT, open source has nothing to do with computer programming where the programming code is made available for anyone to see).
OSINT is an abbreviation of Open Source INTelligence and refers to the knowledge and insights that can be gained by collecting and analyzing data and information from sources that are generally available and meant for anyone to use. See the definition of Open source for details. Today, the largest part of the material produced and used by intelligence and security agencies globally is OSINT – some claim it is 80% of the total intelligence material, other even say 95%. Intelligence gathering methods such as HUMINT, IMINT, SIGINT and ELINT are typically more risky and more costly than OSINT. Further, OSINT is always ethically and legally acquired, which is far from always the case with other forms of intelligence. For that reason, OSINT is used as much as possible, and other methods are used for very specific intelligence needs. In all situations, OSINT provides the background and the initial briefing. OSINT collection and analysis is typically required before more specific intelligence needs can be defined. Those needs could very well be met by additional OSINT material or otherwise by some other method. It may seem strange and unlikely to many people that governmental and military intelligence organizations would see any use and value in the information printed in regular news papers – isn’t that information uninteresting since it is known by everyone already? you may ask. The simple answer is that it is indeed both useful and valuable. In practice, no person or organization is truly aware of any larger amount of the data and information that is published. Almost all of the published information in unknown to each one of us. However, with a well defined intelligence need, an organization can engage in very targeted searches as well as very specific filtering of this massive amount of data and information. Further, the conclusions you can draw from a particular fact will depend on what you already know. This means that what people in general may see as pointless trivia can be a very important piece of the puzzle for expertise on the subject. The break-through for OSINT and a striking confirmation of its value was achieved by Robert D. Steele through the Aspin-Brown Commission, which arranged a kind of competition where the ability of the CIA was measured against that of commercial OSINT-providers engaged by Steele. It resulted in a major defeat for the CIA-team.
Also, as a further illustration of what OSINT is, and what the difference is between information and intelligence, I quote Robert D. Steele: “OSINT is not OSIF (Open Source Information). By definition, OSINT must represent the process of requirements definition, collection management, source discovery and validation, multi-source fusion (at both machine speed and in multiple languages as well as human speed and multi-cultural), and it must produce actionable decision support relevant to a specific decision by a specific decision maker or decision-making group. Anything less is OSIF, not OSINT.” (Robert D. Steele in a review of the book ‘Open Source Intelligence Analysis: A Methodological Approach’)
Business intelligence is the “information for action” that companies can generate by collecting, organizing and analyzing data generated by their own business processes. This means it is different from competitive intelligence, which regards the world surrounding the company.
Competitive awareness is what environmental scanning hopefully will lead to for companies that do business in a competitive space.
Competitive intelligence is the kind of qualified information or “information for action” that businesses can strive to get in order to make decisions that give them an edge or a head start compared to their competitors. Competitive intelligence is almost always OSINT, sometimes with contributions from HUMINT sources, such as off-the-record tips exchanged between people in the industry on trade shows etc. Competitive intelligence can be described as OSINT applied in a specific area (commercial, profit seeking, competitive markets).
Environmental scanning is something that many organizations and companies do in order to stay up-to-date with what is going in their field of business or area of responsibility. Environmental scanning implies a monitoring activity, where typical OSINT sources and channels are kept under surveillance or regular review, for the purpose of collecting relevant information.
Espionage is a term which is legally defined. This means that an activity is labelled as espionage depending on who does the action, where they do it, and who does the labelling. For example, what representatives of the government in a country does everyday is called “intelligence gathering” by that government, while the same actions performed by foreign representatives on the same territory is unexceptionally called “espionage”.
Investigative journalism is a profession where OSINT methods have been used since long before the governmental and military intelligence communities had begun to regard OSINT as a useful and valuable discipline. While investigative journalism often involves getting information from named or anonymous people, which is HUMINT instead of OSINT, there is typically a large component of researching public archives, literature, previous press and so on – all of which is OSINT.
iOSINT - information on Open Source Intelligence 