As the
amount of data continues to grow exponentially, there has been increased focus
on stored data reduction methods. Data compression, single instance store and
data deduplication are among the common techniques employed for stored data
reduction.
Deduplication often refers to
elimination of redundant subfiles (also known as chunks, blocks, or extents). Unlike
compression, data is not changed and eliminates storage capacity for identical
data. Data deduplication offers significant advantage in terms of reduction in
storage, network bandwidth and promises increased scalability.
From a simplistic use case
perspective, we can see application in removing duplicates in Call Detail Record
(CDR) for a Telecom carrier. Similarly, we may apply the technique to optimize
on network traffic carrying the same data packets.
Some of the common methods for
data deduplication in storage architecture include hashing, binary comparison
and delta differencing. In this post, we focus on how MapReduce and…
Comments
Post a Comment