Millions of Shoppers + Scanning Millions of Product Barcodes + Millions of Retail Store Locations + More than 40,000 Retailers Delivering Inventory and Price Information = BIG DATA.

ShopSavvy is now three years old and during that time has become the largest mobile shopping community in the world. Delivering real-time pricing, inventory, reviews and product information to millions of users, for millions of products from thousands of retailers is hard work – perhaps harder than it sounds. Late last year we secured a $7M investment lead by Facebook Co-founder Eduardo Saverin with the goal of making our platform more social. Layering a set of social graph data over our existing product data entailed the creation of an entirely new platform that could leverage the latest ‘Big Data’ tools like Cassandra, Hadoop and Mahout – ENTER ‘ProductCloud’.

ProductCloud is ShopSavvy’s Big Data solution for real-time social product data. Before the release of ProductCloud we used traditional LAMP tools for data – tools like MySQL. By Black Friday we had more than 1 billion rows in MySQL – the realistic limit of the technology. Our team has feverishly been building ProductCloud and I am pleased to announce that it is going live as I write this blog post.

Using Cassandra with Hadoop for data storage and processing, ProductCloud is able to iterate over a MASSIVE dataset on demand while concurrently serving data requests and meeting our various internal and external SLAs for companies like CNET, About.com, PriceGrabber and Walmart. Cassandra’s built-in replication suite allows ProductCloud to maintain load during peek times (such as Black Friday or Cyber Monday) as well as during hardware failures across multiple machines, racks or even data centers. ProductCloud maintains HUGE histories of prices, products, scans and locations that number in the hundreds of billions of items.

These Big Data tools make it possible to leverage the massive amount of data we store in order to constantly create new analytics to provide actionable insights into the retail shopping business. ProductCloud’s open architecture allows our team to layer tools like Mahout on top of our platform to enable new features like price prediction, user recommendations, product categorization and product resolution. What sort of data are we talking about? Here are a few stats that boggle the mind (well at least ours):

-  More than 240,000,000 product pictures and user action shots
-  More than 3,040,000,000 product attributes (color, size, features and so on)
-  More than 14,720,000,000 prices from retailers
-  More than 100 price requests from ShopSavvy users per second

With the release of ProductCloud our team has laid the groundwork for the ‘socialization’ of ShopSavvy. Adding a social layer to such a rich set of data – data that must be accessed on a real-time basis wasn’t possible before the creation of the Big Data tools found in ProductCloud. Look for social features later this quarter.

Early last week we announced our latest feature – SavvyListings in this press release. The idea was simple, let ShopSavvy users scan the stuff they already own in order to list it for sale in our marketplace. With just a scan and a couple of taps you can list all of your unwanted DVDs, video games, books and CDs. The idea isn’t terribly unique – i.e. it is simply a mobile version of Craiglist. The unique part of the feature is that we already have a huge marketplace full of products and shoppers. Within hours of release users had listed thousands of items for sale and the pace of listings has increased dramatically as popular news outlets like TechCrunch, GigaOm, VentureBeat and so on.

To be honest we didn’t realize the feature would get so much attention. We were looking for a simple feature we could release just after the holiday shopping season to tide users over before our full revamp of both the Android and iOS client coming by the end of the quarter. In fact, I was in the Florida Keys speaking at the Mobile Insider Summit and didn’t even have a chance to write a blog post detailing the feature until today.

If SavvyListings continues to be popular with users we have big plans for the feature. I’ll keep you posted. For those of you who missed the release here are the details:

ShopSavvy, the leading mobile shopping assistant, today announced the introduction of “SavvyListings,” allowing shoppers to now sell products directly from their smartphones with a simple scan. SavvyListings will connect sellers with ShopSavvy’s community of more than 20 million mobile buyers, providing both parties with an unmatched mobile commerce experience.

Scan a product and make it available to an audience of 20 million with just a few simple steps using SavvyListings. ShopSavvy will even provide a suggested price for each item to ensure that sellers are accurately listing their products for quick sale. Sellers easily earn money by finding a new home for their unwanted products through ShopSavvy, which has the world’s largest mobile shopping audience.

“We receive constant feedback from the ShopSavvy community asking for new ways to help them save money,” said ShopSavvy CEO and Co-Founder Alexander Muse. “Since ShopSavvy’s inception, our team has focused on ways to help mobile shoppers purchase. SavvyListings not only gives our shoppers a way to find gently-used products at an even lower price, but also provides a way to make some money, too.”

SavvyListings is an easy and free service that makes listing and selling items privately easier than ever before. Users simply scan the barcode of the item they are looking to sell, provide price and condition information, and submit their product for sale. Interested buyers will then be able to see the product listed alongside new products through the ShopSavvy app and contact the seller directly for purchase and delivery arrangements.

With an emphasis on simplicity, SavvyListings does not require sellers to include images, descriptions, categories or shipping costs in their listing, but simply connects them to the buyers looking for the best deal. Shipping and delivery are left to the discretion of the seller and can be arranged at point of sale.

SavvyListings is included in the ShopSavvy mobile app, which is available across every major mobile platform including iOS, Android, Windows 7 and Symbian.

We had some great press today regarding one of our newest features, SavvyListings. The enthusiasm has been overwhelming and we are excited about this feature as well. With the announcement, we wanted to take a quick opportunity to add more information about this feature related to a user’s ability to remove a listing from the ShopSavvy marketplace:

You absolutely can remove the listing on the Android version of ShopSavvy.  This capability is live in ShopSavvy today.

On iPhone, this capability is currently pending review and approval in the iTunes marketplace.

As an early taste, below are screenshots of what to expect in this release:

Just before kickoff we posted ShopSavvy’s prediction for both the AFC and NFC 2012 Championship games. We based the prediction on the barcode scans of each team’s jerseys. We figured that the team with the most barcode scans would win their respective championship. I am pleased to announce that ShopSavvy correctly picked the Patriots and the Giants. Will ShopSavvy be able to use the same method to predict the winner of the 2012 Superbowl? Stay tuned.

Our friends in the media asked us to pore over our data to see if we might be able to predict the winner of the NFC and AFC Championship games. We looked at the UPC of the jerseys of the top four players for each team and then counted the barcode scans for each. Based on that data here are our predictions for who will win the game:

Ravens vs Patriots
Winner: Patriots

Giants vs 49ers
Winner: Giants

At the end of the day fan loyalty might be a great predictor of who will win. We shall see. If this works we will definitely do it for the Superbowl.

This week several of the Android blogs wrote about ShopSavvy Wallet and some asked if it was smart for us to go head-to-head with Google and their wallet. Many reporters have asked me, “Which is better?” My usual response is to ask them a question in response, “Which is better a credit card or a pocketbook?” Google Wallet is like a credit card – ShopSavvy Wallet is like a pocketbook. Both are handy, but they serve very different purposes. Let me try to explain.

ShopSavvy Wallet was designed to make it easy to buy products from within ShopSavvy. Lets say you are standing in Sears. You scan the barcode of a 3D TV you want to purchase, but ShopSavvy shows you that Walmart has the same TV for a LOT less. If your ShopSavvy Wallet is setup you can tap the blue pricetag button and purchase the product without having to visit Walmart’s website. ShopSavvy does everything for you – entering your name, email, phone number, shipping and billing address as well as your credit card information. With one swipe of your finger you have saved LOTS of time and hassle.

Today our wallet will allow you to store any credit or debit card like Visa, Mastercard and Amex. In the future you will be able to store other payment types – you will even be able to use your Google Wallet.

Now let me explain what ShopSavvy Wallet is not – in the simplest terms it is not actually a wallet in the sense Google Wallet is actually a financial wallet. Google’s wallet is powered by Citibank – thus it is not just storing payment information – it is the actual financial instrument. ShopSavvy never charges your credit card – we let the merchants charge your card. Google is in the middle of the transaction charging your card (in most cases they are the card). We aren’t involved in the transaction at all. Google is intimately involved. We don’t get a cut of the merchant fees. We don’t get the processing fees. Google does. Google Wallet really is a REAL monetary device. ShopSavvy is just holding data to make it easier for you to buy stuff.

Check out this short video if you want to see how it works:

Yesterday I shared some usage stats for our new Wallet feature. If you are one of the many users of our Wallet feature you are well aware that you could ‘tap to buy’ or ‘slide to pay’ from a limited number of retailers like Buy.com, Barnes & Noble and Target. To make our wallet really appealing we need to add a BUNCH of other retailers. I am pleased to announce that today we are adding a number of new retailers including:

• Walmart
• Half.com
• Overstock.com
• eCampus.com
• Ace Hardware
• Toys ‘R’ Us
• Glyde.com
• Dick’s Sporting Goods
• Radio Shack
• Sports Authority
• GoHastings.com
• GNC
• Modell’s
• ESPN Shop
• NBA Store
• NHL Store
• PBS Store
• iBuyOfficeSupply.com

These new retailers are now live. We are rolling them out with limited access to their inventory to start, but as we prove they are stable we will expand their coverage to include their full catalogs. This brings our total to 39 retailers available in our wallet. Our goal is to add 30 more before the end of the month (adding about 2 per day).

We just finished our fourth Christmas here at ShopSavvy (still can’t believe ShopSavvy has been around that long) and I thought it was worth writing a post about the differences we see between our Android users and our iOS users. This afternoon I posted a few stats we have gathered from our Wallet feature. One stat that really seemed to bother folks were the user adoption rates for our Wallet feature between iOS and Android users – we are seeing a 10:1 ratio on creation and usage between the two platforms.

First, we need to put our cards on the table. ShopSavvy was one of the first Android apps available and Google awarded us $300K in the Android Developer Challenge back in 2008. Needless to say, we are pretty fond of Google and their team. More recently our friends at Apple have been VERY cool to us as well – featuring ShopSavvy in the iTunes store.

Here are some general observations about our Android Users:

• more willing to beta test new/complete features (good trait)
• more forgiving of bugs/crashes(good trait)
• more willing to give you a second chance, and update their market ratings(good trait)
• less willing to respond to questionnaires and polls (frustrating trait)
• far more likely to report you as a spammer if you send them emails (frustrating trait)
• far more concerned about giving access to various permissions (frustrating trait)
• more likely to request support via email or phone (good trait)

When we released our wallet feature on both Android and iOS we updated our permission request to include the user’s address book. The permission was required so that we could auto-fill the user’s name, email, address and phone number from the user’s own address book. iOS users appreciated the feature and mentioned it in their ratings as a time saver. On the other hand, Android users immediately began giving us 1-star ratings in the Android Market and accused us of ‘stealing’ their contacts. We tried to explain the reasoning behind the address permission, but in the end we just gave up and turned off the permission. Android users would have to manually fill in their name, email, address and phone number themselves. As a result of this ‘fear’ we believe our uptake on Android wallets has suffered (the numbers tend to agree).

Why do iOS users seem more trusting than Android users? Perhaps they aren’t really more trusting in general, but perhaps the ecosystem that Apple has created offers them a sense of security. Apple is very strict about which apps get in the market and updates are scrutinized. Google’s open system – anything goes – forces users to fend for themselves. Which is better? I am not sure, but I really do believe that Apple has made it easier for developers like ShopSavvy to gain user trust more rapidly. On the other hand, Android users are far more accepting of bugs and are willing to help you make your app better – this is very cool and something we do not see among iOS users. Which users are best? The ones that use ShopSavvy, of course.

With more than 20,000,000 downloads – ShopSavvy provides a statistically relevant window into the shopping behavior of Americans (and to a lesser extent Europeans). Each time a user scans a product we store ALL of the prices seen by the user and with more than 40,000 retailers we store a LOT of prices. Based on this data we have learned Amazon has the best price about 6% of the time. This means that 94% at least one retailer has a better price than Amazon. This holiday season we began to see a BIG change in pricing from Amazon.

First, remember that $5 discount Amazon offered users of its mobile phone app? Amazon offered 5% or up to $5 off as many as three items purchased through its price check app on December 10th. We looked to see how Amazon’s prices compared with other retailers on the 10th and found that Amazon was the best price only 3.3% of the time. Translation? Either Amazon raised their prices or other retailers lowered theirs on that day. This is almost a 100% decline in their price ranking.

Once we saw this we began looking at other days. On certain days like Black Friday and Cyber Monday we noticed that Amazon actually did better than normal – between 7-9% of the time they had the best price – a 50% increase in their price ranking. On other days, similar to the 10th, we saw a decline in their ranking.

The best data came from my wife. On two occasions she was buying toys for our daughter Erin from Amazon (she buys everything from Amazon) and she found that Amazon’s prices were 100% higher than local retailers like Target. The one I recall the most was some sort of Barbie Jet which was listed on Amazon for $80 – using ShopSavvy she found the same toy for $40 from Target – a huge savings. I think we won Michele over this year. (note the picture is for the Jet today, the prices have gone up a lot – Amazon is still highest)

So what did we learn? Amazon manipulates their pricing all of the time. It is smart to compare Amazon to BOTH local retailers AND other online retailers – all of the time. You never know who is going to have the best deal – but you can be sure than between 90-96% of the time someone besides Amazon will have the best deal. How do you cope? Use ShopSavvy of course – a neutral price comparison app is ALWAYS the best solution.