Data mining technology, which can scan unstructured data to identify trends or link users, and generate reports that answer analytical questions, is finding new users in both government and businesses, according to the Seattle Times.
Success came for the technology with the development of programs that allow computers to understand content. Some of these are text-analysis programs that diagram sentences like a grade school student, the Times said.
Eastman Kodak uses the technology to identify connections in its own and competitors' patent filings, and Mayo Clinic researchers scan doctors' notes to evaluate various treatments, the Seattle Times reports.
The U.S. government’s demand for intelligence analysis provided the stimulus for the development of data mining technology. Companies backed by the CIA like Attensity and Intelliseek now sell data mining engines and data management, according to the Seattle Times. Intelliseek scans web logs and e-mail list servers and has recently partnered with Factiva, which scans media reports to offer “reputation insight”.
Internal systems like HR, accounting and billing or manufacturing can be scanned to produce a single report using eAnalytics Portal, Accounting Today reports in a survey of new data mining software. Programs that can simplify implementation of data mining include FRx Software, which can connect directly to the user’s general ledger or practice management system, without requiring the user to find someone to provide the connection; and Crystal Reports, which works on XBRL standard language and tags data, enabling data mining programs to work more easily.
The Internal Revenue Service (IRS) uses the Reveal system to detect patterns of criminal activity, analyze intelligence and detect terrorist activities. Installed in February of this year, the Reveal system can query data from multiple sources according to a study issued by the Government Accounting Office (GAO).
“System users develop reports that include query results and graphical depictions of the data,” the GAO reports. The work is conducted by the IRS Criminal Investigation Unit.
The use of personal information in government data mining projects was the subject of the GAO study, which had been requested by Senator Daniel Akaka of Hawaii of the Senate Subcommittee on Oversight of Government Management and the Committee on Homeland Security and Governmental Affairs, according to the Associated Press The GAO report concludes that none of the five agencies they analyzed: the Agriculture Department, The FBI, the Internal Revenue Service, the Small Business Administration and the State Department had fully complied with the Office of Management and Budget (OMB) guidance on privacy measures.
Privacy impact assessments were prepared by three of the agencies but not by the FBI and the State Department. Both agencies are working on these assessments, the AP reported. While the GAO found that all five agencies had taken some steps to secure the data, none were in full compliance with legal and OMB requirements.
State Department employees use Citibank credit cards, but the GAO said they had not determined that Citibank’s security met all federal requirements, the AP said. The GAO report noted the loss of computer tapes by Bank of America in February and said “Agencies that do not take adequate steps to ensure information security risk having information improperly exposed, altered or destroyed.”
A May 2004, a GAO survey reported that federal agencies were using or planning 199 data mining projects, the AP reported. These projects scanned credit reports, credit card transactions, student loan application data, bank account numbers and taxpayer identification numbers.