The amount of data collected at every part of the supply chain has steadily increased, enabling retailers to make extremely informed decisions about what to sell, when to sell it and who to sell it to. The science of collecting these vast amounts of data and trying to draw meaningful conclusions falls under the term “big data.” This vast amount of information quickly becomes complex, unruly and difficult to store; and it’s nearly impossible to extract meaningful insight using traditional databases and computational techniques. The ability to compare Apple iPads to orange t-shirts in a meaningful way is a recent development, and this kind of knowledge is more than power – it’s profit.

Fifty years ago, if you walked into a local retailer to purchase a record player and a copy of Please Please Me, the debut album of a then-unknown band from England, you would have simply walked up to the register with your items, paid the cashier, received a receipt and been on your way (to having your musical world turned upside down, but that’s outside the scope of this blog post). Once a month, every store employee would stay late to manually look through every item on the shelves to take inventory.

Twenty years ago, if you walked into a local retailer to purchase a cassette player and a copy of The Cranberries’ debut album Everybody Else Is Doing It, So Why Can’t We?, you would have walked up to the register, paid the cashier, the now-computerized register would have automatically removed the items from the store’s inventory system, you’d be furnished with an itemized receipt, and you’d be on your way in record time. At the end of each week, the store’s computer would provide the manager with a report showing current inventory levels along with statistics on what was sold during the week. The manager would use that basic data to make informed decisions about what to order and when to order it.

Ten years ago, if you walked into a local retailer to purchase a CD player and a copy of The Strokes’ Room on Fire, you would walk up to the register, the cashier would scan your items and a prompt would pop up on their screen suggesting that perhaps you might want to preorder Radiohead’s upcoming album, Hail To The Thief. You’d ponder for a moment and remember that you used to enjoy Radiohead, so you add it to your order, pay the cashier, get your itemized receipt and preorder voucher, and leave the store, not only excited about your current purchase, but rife with anticipation for the new Radiohead album. At the end of the week, the store’s computer system would automatically place an order to replenish stock on items that were sold, and your preorder would be secured along with those of 50 other patrons – 30 of whom, like you, had entered the store unaware that Radiohead had a new album coming out.

Today, thanks to big data and predictive analytics, retailers like Wal-Mart know what their consumers are going to buy before they even enter the store. When it rains on a Sunday in San Diego California, Wal-Mart knows that on the following Monday they’re going to sell three times as many iPods than normal. They can even identify specific genres of music that will see a temporary boost in sales. Does your business have the ability to translate this big data knowledge into profit?