Wednesday, October 31, 2012

How much data is Bigdata?

It's all relative. What is considered Bigdata varies depending on the capabilities of the organization managing the dataset. For some organizations, facing hundreds of gigabytes of data for the first time may trigger a need to reconsider data management options. For others, it may take tens or hundreds of terabytes before data size becomes a significant consideration.

Amount of data is just one of the key elements in defining Bigdata. Variety in data and the velocity at which data increases are other two major elements in defining a data set to be Bigdata.

Variety in data means having many different data and file types which may required to be analyzed and processed in ways which is out of bounds of traditional relational databases.Some examples of this variety include sound and movie files, images, documents, geo-spatial data, web logs, and text strings.

Velocity is about the speed of change in the data and how quickly it must be processed to generate significant value. Traditional technologies are especially poorly suited to storing and using high-velocity data. So new approaches are needed. If the data in question is created and aggregates very quickly and must be used swiftly to uncover patterns and problems, the greater the velocity and the more likely you are to have a Bigdata problem at hand.

Any feedback, good or bad is most welcome.

Name

Email *

Message *