What You Need to Know About Big Data Analytics Costs
BDA isn’t new anymore. It has a track record, a performance history that we can refer to. By now, we all know the benefits of BDA. But whether you are in various stages of BDA implementation or still on the sidelines, bottom-line costs are probably an overarching concern.
Estimating BDA Costs—Not Quite So Simple
One of the most important ways to justify BDA projects (or not) is a cost analysis. Setting it up is easy, right?
BDA costs = hard costs (TCO) + soft costs.
Well… not so fast. These are the costs of taking action. But did you ever consider the costs of not defining your uses carefully? Or the costs of doing nothing? The real BDA costs are:
BDA costs = hard costs (TCO) + soft costs + cost of inaccurate use cases + costs of not implementing BDA
These additional costs are just two of the real-life considerations that don’t always get a close look before contracts are signed.
So, what questions should you ask before you buy or subscribe to BDA capabilities?
Defining Your Use Cases Accurately
If you want to implement BDA projects without busting your budget, ask: why will we (or are we) implementing BDA right now? What business processes can we improve?
If you mentioned specific processes such as improved security, internal audit or customer 360 data analysis, you’re on the right track. You’ve made BDA development a needs-driven process. By defining your use case precisely, your implementation defines the value you want. And, business results will define your data requirements.
If you work with a vague use case or are implementing BDA for its own sake, get ready for grief. You’ll get unfocused results and find it hard to control the effort and expense of implementation.
Time- and Money-Saving Tip
It’s much more cost-effective to analyze only need-driven data and bring it into a data lake rather than transferring a large volume of data into the lake and then defining its use.
To make this understanding a bit more practical, imagine going into the mall on Black Friday without a clear end goal of what you want to purchase. You’ll end up wasting a lot of time along with missing some better deals on items that you actually need. Whereas if you plan, you’ll know
exactly what stores to visit and what items to look for your optimal shopping experience. Similarly, with your big data project – to make the cost worthwhile for your end goal to improve your business and gain a competitive advantage, it is imperative that you define the problem you are trying to solve.
Choosing Your Partners Wisely
Your next decision is deciding to use inhouse data science specialists or seek external help. If you have certified Hadoop developers or administrators on staff, it helps to get experienced, certified consulting help to define your data strategy or use cases.
Or, if you have experienced data scientists inhouse, it might be helpful to outsource Hadoop cluster administrators to an experienced and trustworthy consulting partner. For at least the first few use cases, consider using outside help to set up BDA capabilities and guide your efforts.
Choosing Your Hadoop Distro
Now that you’ve defined why you’re engaged in BDA and decided on outside help, it’s time to choose the specific Hadoop distribution that’s right for your operations.
Not all Hadoop distributions are priced fully. In fact, price should not be the final word in making your decision. Prices of analogous Hadoop components don’t vary that much between distributions.
While MapR distribution is thought to be expensive, you might be able to ignore unique MapR components such as MapR DB or Cloudera Manager, which are priced separately. If you don’t need the extra capabilities, there’s no need to pay for them. If you do, you’ll probably find that the extra cost is worthwhile.
Regardless of what you pay for your software, license fees are but the beginning of the overall cost of operation for an analytic platform.
A few other costs to consider are:
* Recurring licensing and support fees associated with platform of choice
* Maintenance and support costs of the used hardware where the software will be installed
* Power, space and cooling used by the hardware
* Configuration of security protocols
* The acquisition and ETL of data to prepare for analysis
* Maintenance of the platform
* Development of application programs for ongoing processes
* Test environment
* Training of staff to work the platform [source: https://www.teradatamagazine.com/v13n04/Connections/The-Real-Cost-of-Analytics/]
The Costs of Doing Nothing
But, just as there are costs of implementing BDA projects, there are opportunity costs of staying inactive. Here are the costs of not implementing Big Data analytics.
