Have you ever bought something online and then had that website suggest
related items you might like? That's data mining at work. It's about finding
meaning in a sea of information.
"Data mining is a way for business people to discover useful information
from data, and put that information to work to solve a business problem,"
says Meta S. Brown, author of Data Mining for Dummies. "By business, I don't
mean just commercial organizations. Nonprofits, governments and academics
also use data mining."
People who aren't even called data miners might actually be doing data
"There are a variety of job titles," says Zhen Mei. Mei works at a data
mining company. "Data scientist, data analyst, data expert, data consultant,
data modeler, data mining researcher, data analytics developer..."
Gordon Linoff is the founder of a data mining company in New York. He says
companies need people to turn the data into more solid information they can
use. "A lot of data just sits around in databases, doing nothing interesting,"
he says. "Data mining can turn the data into actionable results."
Social media is changing the way data miners work. "This is an industry
that is still in its infancy in understanding what can be done with its data,"
"Link analysis describes a set of techniques that they use."For instance,
companies would like to know who the 'influencers' are in the community, so
they can target advertising to these people. The government wants to find
criminals online who are up to no good, and link analysis can help with that
So what do you need to know to be a data miner? Linoff says that there
are basic skills like being comfortable with numbers and math. But there's
more to it than that.
"Perhaps the most important characteristic is curiosity," he says. "Often,
you are thinking about something that is interesting, and then you want to
look inside the data to figure out if, when, and how it is true.
"Another skill that is perhaps underrated is communication," he continues.
"I spend a fair amount of time writing books and teaching classes to adults,
so obviously communication skills are important for these endeavors.
"It is also important when working on a project, because you often have
to convince other people about your findings. And they may not be as good
with numbers or computers, so you have to find the right way to express yourself."
But you have to be careful about how you gather and use data. "There will
always be concerns about security and privacy around data mining," says Mei.
Brown admits that "data mining" is already a bad phrase in many peoples'
"In most cases, the fundamental problem is in improper sharing of data,"
she says. "In other words, some people are using data they should not have
in the first place, for data mining or any other purpose. That's why every
data miner should work closely with knowledgeable information technology staff
to comply with privacy laws and good data management practices."
She believes data miners should push for stronger privacy laws. "American
businesses have already lost business with European and Canadian clients in
the wake of revelations about NSA data collection practices. I have also heard
some talk of U.S. businesses changing their data management practices to prevent
government agencies from accessing their data. It will take a lot of effort
to regain the trust that we lost there."
To begin, Brown suggests getting a college education in a field that uses
data to help solve problems.
"Take classes in statistics and other kinds of math," she says. "Basic
programming skills are always desirable, and some roles require a lot more
than basics! If your school offers a class where you can learn to use data
mining software, take it. Develop decent writing and presentation skills."
The Data Mine
This site offers a wide variety of data mining information
Elementary Concepts in Statistics
You'll need to know this stuff to do data mining
Knowledge Discovery and Data Mining
Tips and tools used in data mining