Amazon enters the Big Data game

Posted by Justin Hesser on November 15, 2013

Amazon, the giant online marketplace that sells everything from candy to appliances, has released its entry into the Big Data Games.

The service is called Kinesis, and it's designed to support the efforts of software developing companies. It will be able to stream data instantly, as well as analyze thousands of streams any second. It's also scalable, allowing app developers to pull from as many sources as they need at the time.

As such, it is price a la carte. You pay based on the amount of information used and how it is packaged. The model is based on a unit called a "shard" which measures Internet thoroughput. One shard will allow a user to capture 1 MB per second of data at up to 1,000 PUT transactions per second, and enable apps to read data at up to 2 MB per second.

For those looking into custom database software, this development could be an interesting one: this real-time data will allow companies to react to changes instantly, as well as provide all the other benefits of information analysis. Kinesis might ultimately grant a wider flexibility to businesses with wide user bases. 

In a statement, Terry Hanold, vice president of new business initiatives at Amazon Web Services, described the value of this functionality. 

"Database and MapReduce technologies are good at handling large volumes of data," Hanold said. "But they are fundamentally batch-based, and struggle with enabling real-time decisions on a never-ending — and never fully complete — stream of data. Amazon Kinesis aims to fill this gap, removing many of the cost, effort and expertise barriers customers encounter with streaming data solutions."

While for a smaller company, a service like FileMaker might be the best solution, this large-scale data capturing could be big news for more massive corporations.