Big Data #4: Developing In-house Talent


Big Data without doubt is of strategic importance to any company. It is a hot topic on the corporate IT side, many enterprises are trying to hop on the bandwagon and there are many consultants who want to make money out of the situation. But, seriously what are the long term costs of opening such data to an outsider to catch the hype? What about developing in-house talent, which already know the company and the market?

Many companies that want to implement Big Data and wondering about the human resources fail to recognize that the talent they need is already present. Big Data is all about analytical skills and technology, which means, the power users in your company, who are by default analytical thinkers, and who are also business savvy are excellent candidates for your Big Data team. These users are also comfortable with analysis tools; they just need to learn their way around with the applications.

Once you set up your candidates, you will need to tell them that Big Data is not a side project but a new career track. The team members need to understand that Big Data is a potential game changer for their company, offers a whole rainbow of prospects in every area (both for the company and themselves), new responsibilities and God knows what more. If they accept the challenge, then you really have covered more than half of the road in your Big Data project.

ALSO READ:  Green Data: Is It Catching On and How Does One Comply With “Green”?

Of course in your Big Data project, you will be discussing about the Big Data tool to deploy. Many of the Big Data vendors have similar set of tools, which do not require rigorous training. However, since these are the tools that the team will use to mine, manipulate and report, it is absolutely necessary to spend some time on the tools and choose carefully. Once your team receives the necessary training from the tool’s vendor, it is important to document the processes and procedures in detail. These detailed documentation will help you to keep the knowledge and reproduce it when necessary, without any delay. And these documentation will provide the basis for your company’s in-house Big Data training.

The people, the enthusiasm, the tools are in place and it is time to roll out the company’s vision on the Big Data. If things are not put in perspective, if the things are not put into context that is defined in expectations, roadmap, plans and visions, the whole Big Data project will turn into a huge mess that is not going anywhere (this is not just for Big Data, any project that is not defined clearly is doomed from the beginning). The Big Data team should know these four key points in order to understand what they are doing and why they are doing it:

  • Company’s vision on the Big Data,

  • Company’s project plan to achieve the vision,

  • Company’s roadmap and milestones,

  • Team members, job definitions, responsibilities.

ALSO READ:  Big Data #9: Don't Push It Too Far, Stick to Your Common Sense

Once what, why, how questions are clarified, the next step to get these people underway is training. At the very least you will need to provide the following training to your Big Data team, irrespective of their background:

  • Statistical analysis and modeling

  • Database basics

  • Data warehousing and mining

  • Machine learning

  • Big Data tool the company decided to use – the vendor training as I have just discussed

Training is not something that you, as the company, will provide once at the beginning and then forget later. But rather it is an issue you need to address continuously. Since Big Data is a totally new area for the companies to explore, the companies need to ensure that they are keeping with the latest developments, best practices, new definitions and so forth. I also have to mention that with “training” I mean that both external and internal training. The knowledge and the experience gained in time should be reflected back to the team. In my consultancies, I always advise to send the Big Data team members (also the IT team) to internal trainings. In these trainings they will learn about various business scenarios and will build ideas on how to solve them, which in turn the company will benefit.

ALSO READ:  What Exactly IS "Big Data?"

Once your team is equipped, up and running, you may see your prospective data scientist in your team. The data scientist is a position that is hard to fill nowadays (possibly one of the hardest). Even if you find the right person, the salary and the benefits will possibly be too high. On the other hand, the Big Data vendors will claim that their tools negate the need for the data scientist. Personally, I find these two approaches to be the extreme cases. These budget for the data scientist can be spent on the team as a whole, or maybe a little bit positively biased on the team members who can bring in creative thoughts, different perspectives and shows exceptional performance on their jobs. Spending the budget on your Big Data team makes more business sense; they already know the industry, the business environment and the company’s processes better than anyone else so they provide higher returns on investment.

The outside help that you receive from the consultants at the early stages of Big Data will bring invaluable expertise and knowledge to your company. On the other hand you cannot outsource all your Big Data project to an outside contractor or a consultant. I believe that a little help to get started and utilizing your company’s resources is overall the best way to go in your Big Data project.


  • Featured Image:

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes:

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

2 thoughts on “Big Data #4: Developing In-house Talent