Earlier this month we took our Hiptype.com analytics service offline as we transition Hiptype to a new home where we’ll be able to have a much larger overall impact and achieve our mission of bringing data-driven tools to book publishing.
We have learned throughout the last few months that there are major challenges facing innovations in eBook publishing.
Discovering an Opportunity for eBook Analytics
Back in the spring of 2012, we were just a couple startup geeks at the initial stages of writing an eBook about analytics and data. As we researched the various options for publishing our book, it quickly became apparent that eBook publishing was years behind the web and native apps in terms of the types of insights and analytics available to publishers.
While publishers of apps and websites can now capture sophisticated data about their customers and how they use their products, eBook publishers typically have little information about how readers are engaging with their titles.
In other environments like the App Store and the web, third-party analytics plugins have addressed the demand for data by allowing publishers to easily include analytics capabilities with a few lines of code. It occurred to us that a similar concept could work with eBooks.
We got in touch with a few contacts in the publishing business, and their frustration with the lack of data available was palpable. This was especially true among authors and publishers, and we were surprised to learn that even the largest publishers in the world didn’t have even the most basic data about how their books are used.
Our attention quickly moved from the eBook itself to this fascinating question of whether it’s possible to add a third-party analytics plugin to an eBook document in a way that would work with any of the most popular eBook readers. What if we could help authors and publishers learn about the most and least popular sections in a book, and the various segments of their audience? We simply had to try it.
Implementing the Impossible
We decided to give ourselves a couple weeks to build a prototype for an analytics plugin that could be easily added to an eBook. But this was a daunting task, as none of the most popular eBook formats contained an officially supported way to execute javascript, load tracking pixels, or do anything else that could obviously be used for analytics purposes.
We first narrowed down to a couple of specific eBook formats that were both popular and more likely to yield some way of supporting analytics features. These were the ePub-based formats that are used by Amazon, Apple, and Nook.
The latest ePub specifications contain support for several features that could be used for analytics purposes. And while formats like Kindle Format and iBooks are proprietary, they are modified versions of ePub that are easy enough to work with for a developer familiar with HTML.
Within a few days of intense reverse engineering of these ePub-based formats, we found a way to get the exact kind of analytics data we’d been seeking titles published in Apple’s iBooks format. We quickly started scheduling meetings with eBook publishers to get the analytics service deployed.
A Promising Launch
Within just a few days of completing our prototype, we submitted an application to the Y Combinator Summer 2012 program and within a few weeks we went for a secondary interview and were accepted into the batch.
Our team moved to Mountain View for the summer, and we quickly got to work out building out our service. We created an online dashboard at Hiptype.com, where you could upload a book document and download a version modified to include the Hiptype analytics plugin. Once the plugin was loaded into one of the supported eBook readers, the dashboard would instantly fill up with information about the audience of the book and how they were engaging with the title.
It wasn’t long before word got around within publishing circles and we found ourselves being contacted by executives from the world’s biggest and best publishers.
We launched in the first week of August, with coverage provided by Fast Company, Techcrunch, PandoDaily, Publishers Weekly, PaidContent, and other top publications. Within a week of our public launch, thousands had signed up for an invite to try Hiptype.
Working with eBook Platform Vendors
Unfortunately, the launch spotlight also forced the delicate conversations we’d been having with eBook retailers to speed up. The vendors controlling the largest eBook platforms had some legitimate concerns about a developer ecosystem springing up around their services, which were not designed with this type of third-party development in mind.
The primary concerns we discussed with our contacts at the eBook platform vendors were privacy and performance. If we could address both of these areas, it looked promising that we’d be able to work out a way to get our analytics plugin available to millions of titles on these platforms.
Privacy Concerns
Privacy is a huge issue affecting all types of technology. There is currently little in the way of regulations and laws about tracking, analytics, personalized advertising, and other uses of data that many people feel invades their privacy.
The privacy concerns surrounding Hiptype were not substantially different than those for any other type of analytics product. There needed to be a clear disclosure letting readers know that their eBook contained an analytics plugin, and an opt-out preference that would work across all titles. Additionally, vendors asked that we kept all analytics anonymous, and didn’t attempt to retrieve any kind of personally identifiable information (PII) such as names and email addresses.
We updated the analytics plugin so it contained a disclosure about Hiptype’s anonymous usage statistics, and included a link to more information and a simple opt-out button that would set a preference across all books.
We also made some technical changes to the analytics plugin to make data shared with our servers encrypted for security, and replaced a demographics panel in our online dashboard with a “Personas” panel that would provide the same amount of insight to publishers and authors but abstracted away the exact demographic details to lower the “creepy factor”.
Performance Concerns
The second type of concern shared by the eBook platform vendors was that eBook analytics would make the reading platforms sluggish, reduce battery at a faster rate, and use unnecessary amounts of data bandwidth. These are truly important issues among mobile device vendors, and we took it upon ourselves to minimize the footprint of our analytics plugin.
Over a Redbull-fueled weekend hackathon, we rewrote the analytics plugin code to make it faster and leaner, and the interface to send analytics back to the server was re-architected to batch requests to only send data to the server once every 30 seconds so that less battery and data would be required.
Obstacles and Setbacks
While the conversations with our contacts at the eBook platform vendors initially seemed promising, by mid-September it was clear that getting approval for Hiptype would be no cakewalk. It didn’t help that publishers and retailers were dealing with antitrust lawsuits from the US Department of Justice - there was concern that if any data happened to be shared between parties, it would appear that publishers had formed a monopolistic “data cartel”. While it wasn’t our intention to form such a cartel, we were surprised by how often this was brought up in conversations.
Just a few days after speaking with a representative from Apple, an iBooks update was released that removed the support for analytics we previously had. Progress with Amazon and Barnes & Noble had also stalled. In all three cases, the eBook platform vendors also had corresponding app platforms that publishers have been using when they desired features that weren’t supported by eBooks, sometimes with great results.
The problem is that from all three vendors, we were eventually told the same thing - if we wanted a feature that wasn’t supported for eBooks, then we should be looking at their app platforms instead. We were referred to app store representatives and told that we would be supported as an analytics plugin if we followed the developer guidelines for these platforms.
But our mission wasn’t to provide yet another analytics plugin for mobile apps. These existed already. Our mission was to provide an analytics plugin for eBooks, so that data that was previously locked up could be made available where it was needed most.
Maybe in a year or two there will be more pressure on eBook platform vendors to open up their eBook environments, even a little bit, for third-party services that abide by best practices for privacy and performance. One simple way to do this would be to support ePub3, the new generation of ePub that allows for tracking pixels and javascript code to be included within books.
But right now, there’s no indication that ePub3 or a format with support for similar features will be adopted by the eBook platform vendors within the foreseeable future. With Hiptype, we were too early. And in the fast-paced world of startups where you often don’t have more than a few months of runway to prove out an idea, being too early is the same as being wrong.
Next Steps
While we’re sad to shutdown Hiptype in its current form, we couldn’t be more excited about the new home we have found for the service. We’ll be announcing details shortly.
“(Amazon) has nonetheless created a heatmap of the American election based on the partisan nature of books bought by state.
The company’s methodology is simple: it classifies books into "red” (Republican), “blue” (Democrat) and neutral. Then, it looks at the 250 top-selling books in both red and blue categories over the past 30 days (updated daily) and notes the shipping address. The map’s color and intensity displays which states are turning over pages of what partisanship.
By this measure, President Obama will face troubles on November 6th. As of August 24th, only five Northeastern states are reading more blue books than red ones. The rest of the nation is thumbing red tomes, save for two states that are evenly split.
At Hiptype, we’re fascinated by the idea of predictive analytics. The flavor we are most concerned with is predicting a bestseller. Is it possible to determine the commercial success of a book from what kind of audience the book has and how they interact with the book? How small can the sample size be while still maintaining enough predictive abilities to be useful?
The data that Amazon is using to create these heatmaps is partially public. It’s just sales numbers cross-referenced with where those books shipped - which is information that only Amazon has.
It might be more interesting if Amazon were to not only include sales data, but also to include reader engagement. It’s one thing for most of the books shipped to Ohio to be of a “red” nature, but what if the overall amount of engagement for political books in Ohio has been greatest for “blue” books?
There’s still a few months until the election. Let’s hope that Amazon is willing to drill down and actually see if there’s something to this idea of book analytics predicting political outcomes.
We were interviewed by Joe Wikert in O'Reilly’s fantastic TOC Podcast series:
What exactly is data-driven publishing?
It goes beyond simple sales stats and review information to understanding how the product is used; where readers spend the most time; and even though we don’t like to think about it, how far they get before they abandon a book.
Behind the product, Levy and Prasad have an ambitious vision.They argue that e-book publishers need to follow the example of popular websites by using data to become leaner and smarter. For example, they say publishers could start testing their books by releasing them to a small number of readers, then tweak the content based on the data. Hiptype also allows publishers to use its data to create targeted Facebook ad campaigns.
“We believe that data will save the book publishing industry,” Levy says.
Publishers testing Hiptype in beta, for instance, were surprised by “how low conversion rates are” — early data suggests that only three to four percent of people who download a free ebook sample go on to buy the book — and how few people who do buy a book finish reading it.
“It can be a bit of a bummer,” Levy said. “But as soon as you start measuring, you can do tests and see what moves the needle. We’re already doing research on the data we’re collecting. As data hackers, we think there are underlying patterns here even if they’re not apparent at first.”
We couldn’t have made Hiptype without the outstanding advice and support of our advisors, our YC mentors and batchmates, our friends, and our families.
The trilogy has combined to sell over $20 Million in less than five months. By comparison, Stieg Larsson’s The Girl with A Dragon Tattoo trilogy took over three years to sell that many books.
E.L. James’ trilogy is a juggernaut. It’s been ranked atop the New York Times’ best sellers list for 22 weeks. Its sequels, Fifty Shades Darker and Fifty Shades Freed, are currently ranked No. 2 and No. 3. A compilation of all three books ranked No. 9. The movie rights to the series have been sold for millions of dollars. These jaw-dropping sales figures are the pinnacle of a historical rags-to-riches story that touches on the emerging phenomenon of self-published fan fiction.
And while the rags-to-riches angle has undoubtedly helped the book get more attention, the same could be said for a wave of controversy that has surrounded the books due to their overtly sexual themes that revolve around a racy relationship between a submissive college student and an older, domineering man with a mysteriously dark side. The books are filled with graphic NSFW passages like this:
“Hmm… he’s soft and hard at once, like steel encased in velvet, and surprisingly tasty.”
I’m not here to judge. We all have our guilty pleasures. But it’s fascinating - why does Fifty Shades of Grey succeed where so many other seemingly similar romance books haven’t? Is it the equivalent of winning the lottery, or is there an underlying pattern?
The publishing world is running off a cliff, like Wile E. Coyote suspended in mid-air. Borders went out of business, Barnes & Noble is winding down its brick and mortar presence and talented authors, editors and publishers are bemoaning the end of reading as we know it.
Yet here we find Fifty Shades of Grey , a book that is essentially fan fiction, and yet it has captured the imagination of millions of people and has become a part of our cultural zeitgeist. What I’m wondering is, “what can we learn from this book? What might books like this one tell us about the future of publishing?”
Fifty Shades of Data
eBooks are the future of book publishing. This is hardly a bold claim. Anybody can look at sales numbers and see where things are heading. In 2011, eBook sales surpassed print sales in all trade categories, and eBooks are now the most popular format for adult fiction titles.
However, the publishing world can’t fully optimize eBook sales without truly understanding the digital market. Publishers don’t know the DNA of a successful book, and they don’t know how to build an online marketing campaign without the right data. Thus far, publishing has been ill equipped to move fast enough to adapt and prosper in the digital age.
Imagine running a business, and not knowing how customers use your product. Imagine running a business, and not knowing why your customers share your product with their friends, and how that predicts future sales. This is essentially the publishing world in 2012.
When I think of a future model for book publishing, I think of the model created by The Huffington Post. They fully embraced data-driven publishing for the news, with key data-driven team members like Ken Lerer, Jonah Peretti and Paul Berry. They were innovators in Search Engine Optimization. They split tested headlines and photographs. Their technology team build custom analytics tools to track traffic sources, time on site and bounce rates. They quantified reader engagement to determine the value of each visitor.
The Huffington Post has the DNA for a successful news website, and this helped them create a successful business model for news in the 21st century. Others said it couldn’t be done, and HuffPo proved them wrong.
Fifty Data Points Freed
Publishing needs a little bit of that same DNA that encourages a “hacker” culture, where data wins arguments. The publishers that figure this out will be tremendously successful and profitable, and the ones who don’t will go away.
If book publishers had a way to capture relevant data about who is reading their books, how they interact with the books, and what persuades readers to buy a book or talk about a book to their friends, they’d be able to produce successful books instead of rolling the dice when they publish.
This would allow them to create more blockbuster successes, and thus create a more successful publishing ecosystem, one that has the money to support important scholarly works or artistic achievements that don’t make a ton of money but create a bunch of social currency.
I believe we’re at the forefront of a new digital publishing revolution, one in which data-driven participants will be more successful, readers will be happier, and ideas will be more easily shared.
And I for one can’t wait to discover what exactly it is that makes up the DNA of a successful book, and finally understand why it is that everyone is going so nuts for Fifty Shades of Grey .
Alexander does a great job illustrating the coming revolution in ebook analytics, and also the conflict it is creating in the publishing world.
Right now publishing is where online ads were in 1999, or where mobile games were in 2006. There’s a lot of new ways to track how readers engage with books and what that says about their customer persona.
Publishers and indie author have access to information that’s never before been able. The question is: which publishers and authors are going to take advantage of all this amazing new data, and which ones are going to fall behind?
There are many early adopters in publishing. They look at the sales figures, and know that the future is in ebooks. So they’re looking for ways to optimize ebook sales. And the best way to do this is to use powerful data to market the right books to the right customers.
Unfortunately, there are some, ahem, old-schoolers who are skeptical about all these new metrics. We’re not exactly sure why. If you are in business, don’t you want to know how your customers are using your product? Don’t you want to know what they like and don’t like, what they share with their friends, what are their key demographics, and what’s the conversion rate? Obviously, the answer is yes.
As we said, ebooks are where websites were in the late 90s or where mobile games were about 6-7 years. There’s a disruptive movement toward data-driven marketing, and it’s just a matter of time until it transforms the publishing world.
We hope the early adopters win this battle sooner than later. We love books, and we want to see publishing thrive. But for that to happen, publisher and authors need to stop worrying about “ebooks reading us” and realize everybody has a lot to gain by better understanding how readers read. Better data means better marketing, more sales, and happier readers.
In Joe Wikert’s newest post, “Why Aren’t E-Reading Devices Smarter”, he touches on a few interesting points - why are eBook interfaces skeuomorphic?, for instance.
Don’t let me forget about samples. Sample content management is pathetic on all the major ebook platforms. Seriously. I’ve told B&N that I’m interested in a title and they’re content to simply toss the short sample my way and never follow-up. I’ve got samples that are really old now and I’ve forgotten about them. Let’s have a feature in this manager that knows when I downloaded every sample and, based on how I configure it, reminds me to check them out….And let me subscribe to samples! I love baseball. Go ahead and send me the sample for every new baseball ebook as it’s published.
Based on the data we’re collecting, we’re seeing that the rate of growth for eBook free samples is rapidly outpacing the rate of growth for eBooks, and yet the conversion from eBook samples to paid eBooks is typically in the low single digits. We agree with Joe that the retailers could add a few simple new features that would be game-changers for generating new demand that will translate into substantial revenue for authors and publishers.
There are two basic models of delivering software and content - the “ownership” model and the “access” model.
The “ownership” model only became the norm after years of delivering software and content via shrink-wrapped physically distributed packages. But as digital distribution rapidly overtakes physical distribution channels, subscription-based business models are completely transforming entire industries.
Gaming, for instance. Guess what they’re talking about at the board meetings over at EA and Activision? It’s probably something along the lines of “how do we convert every single one of our customer transactions from a one-time purchase to an ongoing subscription?” As the CEO of Take Two Interactive said last year:
“the holy grail of our business is to take a packaged goods release
and turn it into a subscription model.”
Microsoft’s Xbox, World of Warcraft, and Valve Software’s Steam have demonstrated the massive potential of subscription models. Facebook and all the major mobile platforms have responded to demand from their developer communities by adding support for app subscriptions.
Subscription models are appearing just as rapidly in all varieties of software, as the cloud overtakes even heavyweight flagship professional tools like Adobe Photoshop. And in the music industry, the rise of Spotify, Rdio, 8tracks, and many other streaming music services suggest that the iTunes “ownership” model that was so revolutionary just a decade ago is already behind the times.
And the “access” model can even apply to physical goods, as disruptive companies like AirBnB and Zipcar are proving.
One of the biggest benefits of the “access” model aside from the consistent and predictable revenue stream, is that subscriptions make it far easier to launch with a first revision, see what works, and continue iterating and testing different ways to improve and expand the product.
And it works - the subscription model has resulted in gamers buying a new title from Blizzard expecting that a year later, they’ll have a game with new features, more levels and characters, and other types of enhancements that will make the product a gift that keeps getting better.
One of the things we’re really excited about at Hiptype is helping book publishers go through a similar transition from a strict “ownership” model to an “access” model where performance changes between iterations can be easily measured, and in-book revenue streams help compensate for downward pressure on retail prices.
Since I was about 12, it’s been a dream of mine to write an adventure game like the old Lone Wolf books by Joe Dever. Thanks to some encouragement from my publishers at Breadpig, I finally did it.
Imagine how much better a choose-your-own-adventure book would be if the author and publisher could measure which paths its readers most often take, and then add more content to expand on those paths just as Blizzard might add an expansion pack to the sections of WoW most popular with its users.
Sure, we’re not used to thinking about books this way, with some notable exceptions such as O'Reilly Media’s Rough Cuts. But as we’ve seen with all kinds of other forms of content and software, if the benefits are great enough we can quickly adjust to the new standard operating procedure.
Our vision for Hiptype is to shorten the amount of time it takes to the point where we look back and wonder how it was that we ever published and read completely static, unchanging books.
