This article is part two in a series on how to transition your technical career to include big data. In the first article, I described the true value of big data in terms of gaining new insights about customers, products, partners, supply chain, and other important contributors to business success.
Now let’s dive in big data processes and architectures.
Big Data Processes
The processes for big data adoption begin with the business case—making the connection with value as the fifth V of big data. A sound business case needs to identify business needs and opportunities, areas of insight to be gained from big data, expected outcomes from new insight, and a projection of value derived from the outcomes.
A sound technical case should follow the business case. The goal here is to avoid a tendency to pursue big data as the next bright, shiny object. Consider alternative ways to achieve the desired outcomes and rule them out only on the basis of sound rationale. Also consider whether you have people with the right knowledge and skills to do the job, ask about the implications of big data sources for your data governance processes, and consider availability and quality of the data that you’ll need.
With business and technical case in place, think about each process that works with data. How are the processes performed today with your internal and structured data? How might they need to adapt for external, unstructured, and streaming data? Consider processes for:
- Data sourcing including data selection and connectivity.
- Data preparation including selection, cleansing, integration, dataset reduction, and data storage.
- Data analysis including problem framing, analytic modeling, and data visualization.
- Data usage including consumption by people, business processes, applications, and data stores such as a data warehouse.
Big Data Architecture
Architectural integration is as important as process integration. You already have BI architecture structures, standards, and conventions in place. Big data will not necessarily change the existing architecture, but it will extend and expand it in all of these areas:
- Data storage – expanding from structured data stored in relational databases to encompass document, graph, geospatial, and NoSQL databases.
- Data access – expanding to access and acquire streaming data, device data, web data, social media data, and more.
- Data consumption – expanding to include big data visualization capabilities and possibly to extend the data warehouse with time-variant big data.
- Data management – expanding the scope of data governance and data integration, adding complexity to data quality, and amplifying the need for data virtualization.
- Analytics – with increasingly complex and higher visibility analytic models for machine learning as well as predictive, prescriptive, and descriptive analytics.
- Technology – integrating Hadoop, NoSQL databases, new data access technologies, and more into your existing technical architecture.
Getting Started with Expanding your Technical Career to Include Big Data
Readiness assessment is a wise first step before undertaking the initial big data project. Formally or informally evaluate each of these success factors:
- Do you have known big data opportunities?
- Are the business goals clearly stated?
- Do you have active business sponsorship and engaged stakeholders?
- Have you considered the risks of the project?
- Do you have mature data management practices that are ready to extend to encompass big data?
- Is your organization skilled with iterative development?
- Do you have the needed technical capabilities?
Asking these questions helps to prepare for the steps of project planning and execution. On the surface they are much the same as what you will do for any BI project, but big data adds a few twists particularly in the areas of project risk and the need to iteratively discover requirements. And you’ll certainly want to plan post-project activities to capture lessons learned and begin the process of maturing your big data capabilities.
You can solidify your technical career, with online education options. You’ll find valuable Big Data education at eLearningCurve. http://ecm.elearningcurve.com/ProductDetails.asp?ProductCode=BD-01-a
For part one, see Big Data and Technical Careers.
About the Author – Jennifer Hay
“My clients tell me I have the rare talent to transform a technical career into a clear, concise, and powerful technical resume. In fact, I’m known for innovative resumes that provide an advantage in today’s intensively competitive global employment market. As the world’s first nationwide resume writer for information technology (CRS+IT), I am a natural choice for technology professionals seeking high-impact career marketing documents.
I won a Toast of the Resume Industry (TORI) award in the technical category in 2011, a Career Innovator’s award in 2012, and served as a TORI judge in 2013.