There are tremendous job opportunities when you expand your technical career to include big data. As the use of data and data management practices continue to evolve, it’s a natural fit. In this article, I’ve outlined some of the fundamental big data aspects that you’ll need to understand and apply.
Adoption rates and success stories for big data continue to lag behind even with all the marketing hype. I believe that the answer is really quite simple—big data is hard because it is a mix of opportunities and challenges. The opportunities drive excitement but the challenges inhibit adoption. The three V’s—volume, variety, and velocity—are widely accepted as defining characteristics of big data but they don’t really describe the difficulties.
The challenges begin with two additional V’s—veracity and value—that are essential to achieve business benefit from trustworthy data. They continue with the complexities of planning, processes, projects, architecture, and technology. Big data brings challenges in each of these areas, largely due to the need for integration.
Big data doesn’t need new planning; it needs to be integrated into existing plans. It doesn’t need new processes, but integration into and adaptation of existing data management and analysis processes. Big data projects need to fit into the current BI roadmap, not to be mapped separately. Architecture and technology for big data must fit neatly into existing BI architecture and technology infrastructure.
The most important thing is what we do with the data—access it, analyze it, and act upon it.
Understanding that big data is really about analysis, action, and outcomes leads to rethinking the definition of big data. It is commonly defined as “data sets whose size is beyond the ability of typical database software tools to capture, store, manage, and analyze” (The McKinsey Global Institute, 2011). But if size isn’t the most important thing about big data, then it should not be the focal point to define big data.
I propose a new definition that focuses on the important elements of analysis, action, and outcomes: Big data is a collection of data sets (often not traditionally structured) that look beyond enterprise boundaries to gain external perspective, increased understanding, and new insights about customers, products, partners, supply chain and other important contributors to business success.
Working from a definition that is purposeful and outcome oriented, we now have a solid basis to extend existing BI processes and architecture to include big data.
The next article in this series examines big data processes and big data architectures. I’ll provide information about how you can leverage your existing skills to transition to a technical career in big data.
Click here to read the article Big Data Skills.
Jennifer Hay Bio
“My clients tell me I have the rare talent to transform a technical career into a clear, concise, and powerful technical resume. In fact, I’m known for innovative resumes that provide an advantage in today’s intensively competitive global employment market. As the world’s first nationwide resume writer for information technology (CRS+IT), I am a natural choice for technology professionals seeking high-impact career marketing documents.
I won a Toast of the Resume Industry (TORI) award in the technical category in 2011, a Career Innovator’s award in 2012, and served as a TORI judge in 2013.