I can't has cheezburger

Media companies have been in a frenzy trying to figure out how they’re going to make money in the age of this newfangled “internet” thing that’s been around for the past couple of decades.  I’m a pretty benevolent guy, so I’m going to just lay it all down here for media companies to copy down. And when I say “media companies,” I’m talking about the networks, the ones you buy entertainment from as part of your cable subscription (or maybe you don’t because you live in the 21st century).

Media companies, you’ve been blaming the internet for all of your problems lately.  The more time people spend being entertained on Facebook, the less time they’re spending watching your TV shows and movies. Not making the revenues you used to? “Blame piracy!” you say.

Piracy has several distinct causes. Sometimes people just can’t afford cable but they want to watch your shows anyway. Some people don’t like the ads. Some people don’t want to wait for episodes to end up on Hulu. Some can’t legally get your show in their country in a reasonable time. In all of these cases, there’s something you could be doing to have these customers’ business, but you choose not to, and instead you complain that they aren’t paying you money on your terms and that you’re stuck living a life of semi-luxury. Maybe copyright infringement of your content is morally wrong, maybe it’s not. Who gives a fuck? What matters is that you’re not making money and you could be making more if you’d shut the fuck up about how pirated content is the source of all your problems and maybe find the business opportunity. Below are steps you can take to correct this and prepare yourself for the future, so if you’ll stop being a little bitch for a minute, maybe you can get in on this action and maybe start seeing some growth again. Or don’t, and when you go under the world will wonder what could ever replace your unoriginal content.

First off, fuck the cable companies that you’ve got these longstanding relationships with. This is going to hurt, I know. They’re probably paying you some really good deals to have exclusive distribution rights to your prized TV shows, and to sweeten the deal they’re probably also taking some of your shit content too, and they pass it all off to the consumer in a “package.” Today, you as a media company are making most, if not all, of your money from these cable companies. It’s going to be really painful to walk up to them and say “nope, you’re not going to have exclusive rights to our stuff.”  Before you continue doing what you’ve been doing, which is continue to lose viewers to other sources, let’s rethink the relationship you have with cable companies.  It used to be that the cable company made its money from customers from one source: selling cable TV to subscribers. But since then, they’ve been capitalizing on this period of time during which they’ve been double dipping, collecting cable fees and internet fees. Well, you’re going to ultimately put an end to this by gradually negating customers’ needs for cable and you’re going to directly sell them the content, cutting out the cable company as a middle man.  The cable company’s going to hate you over this. It’s going to kill off a big part of their business over time, and they’re going to have to get used to that.  They’re going to have to downsize and they’re going to have to shift their investments into their internet infrastructure. Ultimately, they are going to thrive as the ISPs of the future, and they’re going to be better at it because then they’ll be focused exclusively on giving customers internet access. They’ll work more aggressively to extent their reach to more places, and they’ll look to investing more heavily into their backbone infrastructure as they seek to get more revenue per user by selling them more bandwidth at higher speeds.

The cable companies are going to fight you tooth and nail. They may even try pulling some shit in which they try to throttle their customers from accessing your content. It’s at this point that you call up the congressmen whose campaigns you contributed to (quit with your surprised looking faces; I know you contribute handsomely to their campaigns) and tell them to take swift action to protect your industry. Believe me, the consumers are going to be with you on this, and they’re going to love seeing this new Hollywood that is an advocate for consumers.

So what are we fucking cable companies over for? Because I don’t have cable and there doesn’t exist a legal way for me to watch new episodes of Shameless when they air. Because I’m part of a new generation of people who grew up with high speed internet access and we’re smart enough to figure out ways to entertain ourselves without paying a $100 triple play cable bill, yet here I am, wallet in hand, looking for a way to see popular TV shows first run, and you won’t fucking sell me the show. Why would you, as a content creator, ever want that? Rectify this. Start offering first run TV content online. Offer it so that everyone who wants to see the content first run can see it at a great price. Go to Apple and have Apple sell TV show episodes to users at 99 cents per episode. None of this variable pricing bullshit, just sell it to them at a flat price. Tell Apple that you want an 80% cut. Apple’s going to balk at this, but you know what? Fuck you, Apple. I don’t see you using your $90 billion stash of cash to create content that’s better than mine, and you’re not going to, because that’s not what you’re good at. You know what you are good at? Making a great device that will make it a joy to experience MY content. But you’re not going to sell many of those devices without my content, so you can take 20% or you can go penetrate yourself with a buttonless Apple remote control and admire how smooth it feels going in. But the key is that you put first run content in the iTunes store (and when I say first run, I mean first run, none of that east cost/west coast delay bullshit), and Apple will make up that lost 10% in volume, and they’ll sell more devices, and they’ll be happy that you made the deal.

For some reason you media companies have been so goddamn uneasy about selling TV shows online at a good price, instead looking to keep stuff on Hulu for free. You’re worried that selling at 99 cents an episode will devalue the content.  Know what? you’re a fucking moron.  I don’t know where you got your MBA and what crackpot theory they taught you there about your content, but advertisers are paying you something like a $20 CPM on your content. I once read that Hulu gets you a $60 CPM, or at least it was before Hulu started showing twice the advertising in a show.  Selling your episodes through Apple at my aforementioned 80% cut of the retail price, you get an $800 CPM.  That’s fucking forty goddamn times the CPM of what you’re letting advertisers pay you for that content. And in addition to making 40 times as much money, your TV show isn’t interspersed with some stupid ass animation of an army general trying to sell car insurance.  Let’s be honest here, what really devalues your content?  Assuming it’s a 23 minute show, customers pay a little over 2 bucks to watch an hour of commercial-free content. That’s $2 for 14 minutes of commercials saved, which means that as long as your customers’ time is worth more than about $8 an hour they’re coming out ahead.  Of course, most of your customers are too stupid to do that math (because if they were smarter, they’d have stopped watching House after the first season, realizing it’s the same episode each time) but 99 cents will feel like a great price for the show. 99 cents is within reach of a ton of people.

First run content at cutthroat prices like that is going to sell like goddamn gangbusters.  People who previously were blowing $100 a month on their cable bill on hundreds of channels they didn’t want, to get the few shows they did want, are now going to be more than happy to spend their money readily to get first run content. Now that they have a little more money to blow, you damn well better let them do it.  Maybe they want to watch this one old obscure movie that isn’t on Netflix.  Back in the days of physical media it was costly to distribute those movies and give them shelf or bargain bin space, so it made them more scarce. You don’t have that limitation with digital media. Get your entire catalog out there, available for purchase on the cheap. Again, shut your fucking mouth about devaluing content by making it ubiquitously available. We’re talking about your back catalog here. Any scratch you can make on these movies is better than the nothing you were probably getting before with your DVDs in a bargain bin at Walmart. Sell these babies for $3 or $4 a pop and you’ve got a sizable revenue stream coming in from old movies.  Let Apple handle the selling of the movies and allowing people to keep their copies of the movie.  Don’t try to pull some bullshit where the movie can only be watched a said number of times or will expire after a certain period of time. Customers aren’t buying it, and you’ll just piss them off.  If I bought Forrest Gump and I’m sitting on the shitter with explosive diarrhea in a bus station and decide I want to continue watching it, then don’t fucking judge me, let me watch my movie and shit in peace, crapdammit. If I later buy a new iPad or whatever and want to watch it again, then don’t give me any fucking flack about it. I buy it, it’s mine to keep.

And don’t forget to sell your first run movies too!  As much as I know you’d love to try to sell me a digital download of a movie that’s only in theaters for $40 because you think I’m desperate enough to pay it, think again. It’s not only in theaters, it’s also available on The Pirate Bay.  I’m fine with paying $10 to watch the movie that early on. Don’t try to fuck your customers over with this artificial scarcity of limited release windows when the movie’s first out, because if you do that you’re a fucking moron. If someone wants to watch the movie and is willing to pay you money, let them fucking buy it.  Are you sad that the movie theaters will go out of business with this new model? Fuck them.  Their popcorn and ticket prices were obnoxiously overpriced and the previews were a goddamn half hour long (which I know was actually your fault as a media company, so fuck you too). You might have to start offering the content to movie theaters at a better price so they at least have a fighting chance, but with their concession stand profit margins they honestly could probably all retire now.  Perhaps they could start selling their old 100 gallon drink cups to factories to use as barrels.  I don’t know what they’ll do. They can be creative if they want to survive. Maybe appeal to some new markets. It’s not your job to be benevolent (and if it is, your customers sure as shit don’t get that vibe).

It all comes down to the fact that you’re still trying to run media distribution as if the internet doesn’t exist and that video content is a physical good that you can easily control supply of, and the fact that the internet exists is really starting to damage that delusion. You can accept this reality and do your best to try to make a profit from it, or you can continue living in denial, only throwing the tiniest of bones to Hulu and the like to make it seem like you’re interested in the future while you continue to let emerging entertainment forms start to eat away at your customers’ attention spans.  If you’re not willing to sell people your content on the internet, someone else is going to start making content and they will be willing to sell that content, and their shows are going to gradually start appearing in conversation at the water cooler.

I’ll just say it right up front: NoSQL is a bit of a fad. As much as developers don’t like to consider themselves victims of fashion trends, there is this unwritten desire to be working in the vogue languages and technologies, and it’s not at all uncommon for a developer working in these technologies to snub their noses at what were probably perfectly serviceable choices just a few years ago (and still are today).  Oh, you’re writing your software in Java? You’re using SVN still? Omigosh, why don’t you just use Git? Jeez, you’re using SQL? How are you ever going to scale with that? Rails developers had a reputation around here for being totally insufferable when it comes to those using a non-Rails stack, and even they have fallen victim to obsolescence as node.js becomes the new kid on the block.

That’s the sort of attitude that caused me to approach Mongo with skepticism, and you should too. It’s not right for everyone or every data model, and if you have a pretty good SQL based skill set and you know how to tweak an SQL database for maximum performance and/or you can easily model data in a relational way to solve a problem, you may not stand to gain much of anything at all with Mongo.

I’m not an SQL whiz, though. The first DBMS I learned how to use is an old school hierarchy based database used for electronic medical records. Knowing the limitations and annoying issues that this type of DBMS has, I knew Mongo doesn’t make life a cake walk, but having now spent a few months with Mongo-backed applications, I really love it. I don’t have to be a DBA to be able to work with a highly powerful and fast database, and knowing a few simple concepts I can do just about anything I need to in Mongo.

First off, let me explain the anatomy of a Mongo database. At your most general level, you have a database which contains zero or more named collections. Each collection contains zero or more documents, each of which is structured in a binary form of JSON called BSON. Documents in this database can take have all the data types you expect in JSON and can be nested arbitrarily.

Although most of the documents in a collection tend to be pretty homogenous, there are no hard and fast rules about the keys each document in a collection must contain. Also, Mongo has no rules for what data type a particular key should have. If you’re not familiar this would be referred to as schemaless in database parlance.

I think of schemaless databases as being analogous to dynamically typed languages, sort of (like how Ruby is). The flexibility that offers is incredible, especially if you want to quickly get a prototype out, or if you want to quickly start storing your data in a different way. Of course, those schemas were also doing things like preventing you from doing stupid things like deleting a record that was still referenced by another record. Mongo will happily let you shoot yourself in the foot, leaving greater potential for errors at runtime (the worst time for errors). Of course, using an ODM like Mongoid will go a long way to helping save you from yourself. You could implement callbacks that, on destroying a record, will go and delete the records that are pointing to it.

If you learned databases in college, you probably were learning primarily about relational databases. These learnings you’ve acquired have inevitably caused you to reshape the data you think of to fit into this relational sort of mold. You might think to use some collections to join other collections as you might a join table. It’s a trap!

If you’re designing your schemas like this, are you doing it out of habit? If so, stop! If you’re designing the schema that way because you know you’re going to need to report on the data in a lot of different ways later, there’s a decent chance you might be better off with an SQL database.

So, always embed everything, right? Well, not so fast. Although you want to ultimately keep yourself from extra queries just for handling data with relationships, always embedding everything can have bad side effects, too.

Say, for instance, you’re making a database schema for the proverbial blog that has posts and comments on the posts. If you embedded everything, you would have a collection of posts, and each post would contain zero or more comments. It makes opening a post a snap. Just one query and you’ve got all the data you need! However, there’s a catch. Mongo documents are limited to 16 megabytes. If your blog gets on Reddit, you’re going to hit that limit. Or, if you ever wanted users to see a list of all their comments they’ve written, you’d have to cycle through each post and query the post for just the comments you want. That’s expensive and it’s more work for you as a developer. You could instead embed an array of IDs of comment records kept in a comments collection. Or keep them in the same collection; after all, Mongo doesn’t care! Or you could also just use a relation and let each post have many comments and each comment belongs to a single post and will contain the object ID of the post. The more I use MongoDB for production stuff, the more I’m learning to have many classes in my Rails apps (which map to collections) and I give these classes a single role that they can do well and I define their relationships. Yes, that sounds an awful lot like a relational database, but it’s not quite that way. Instead it is closely coupling my model classes with the way they’re actually modeled in the database, and I think that’s great. But it is important to use moderation in how you do that, because you don’t want to have to send the database two dozen queries just to get information for a single page load.

If you’re using Mongo as the primary DB for your web app, you’re inevitably going to end up needing relations, so don’t shy away from them just because you’re not using a relational DB. Just be smart about them!

Another concept that’s going to feel like a sin when you start doing it in a Mongo database is denormalization. Let’s say you’ve got a collection of posts by different users.  What would you store to identify the user? In a relational DB you’d store a foreign key to a Users table. Then, when you got the blog post info, you’d look up that record in the users table and get the user’s name and such.  In the Mongo world that’s a huge waste of time, and you just store the name and any other user info you need right in the post there. You should also include a reference to the user record for discreteness (because you never know when you might be running some data conversion that would benefit from just having an ID), but the info you need to display the post is stored right in the post.

“Omg,” you say, “but what if that user changes their name someday?  What will we do then?”

The people who designed Mongo presumably thought long and hard about this, and decided that although it’s nice and elegant to only have to change a name in one place when a name is changed, just looking at a post happens many orders of magnitude more often than name changes. Therefore, it makes no sense whatsoever to add an unnecessary query every time just to accommodate this. What if a user changes their name? No big deal, just run a batch job that will find documents referencing the document with the name change, and update the fields. It’s expensive, but it won’t happen often. If you’re dealing with the kind of data where that does happen often, maybe SQL is for you. It’s better optimized to jump through hoops to do that sort of crap.

Mongo is also, as they say, web scale. Because of the lack of things like joins which are difficult to scale across many shards, it’s not at all complex to scale your Mongo database across many nodes. 

But it’s not the web-scale-ness or the general performance that makes me so enamored with Mongo. The beautiful thing about Mongo is that it’s such a simple database.  Relational databases are a very unnatural way to think about structuring your data, and their advantages only show up in some scenarios.  With Mongo, you can store the data the way you think about it. Got a record that’s going to contain an array of values as one of its properties? In Mongo that’s trivial. In SQL, not so much (yeah, yeah, Postgres can do it). Want to include some key-value pairs in your document? Easy. Oh, you want to embed a document in your document? Well, yo dawg, you’re in luck, because Mongo lets you do that.

(I couldn’t resist)

Mongo lacks a lot of the complexities inherent in the RDBMSes that have been the standard for the past couple decades. Of course, sometimes the complexities are desirable. For instance, transaction support might be a complexity, but if, for instance, you’re Evernote, and saving a note involves a complex series of all-or-nothing changes to your account, you want transaction support (after all, it would suck if your upload quota increased after you failed to upload a note once).

This lack of all the bells and whistles leaves us right now with a very simple database, a sort of iPhone-like re-imagining of what a database can be and how simple it can be. Some features are going to get added back in, in a re-imagined way (I’m pretty excited about the aggregation framework myself) but many won’t.

 Mongo has some caveats you should be aware of:

• Mongo is a memory-mapped database and it stores its data with the wishful thinking that it’s always completely in RAM. You can’t tell Mongo what to keep in RAM (like, say, just your indexes) if your data set gets bigger than how much RAM you have; you just have to rely on the OS to be smart about caching. SQL databases are a little wiser about this.
• There’s a global write lock that will probably get you bummed out if you do a lot of writes. 
• The database doesn’t have built-in transaction support. If you want to do a set of actions on a database that should be performed in an all-or-nothing way, you’ll need to implement your own logic to undo the previous changes if some later step should fail.
• If you use long, descriptive keys in your database, Mongo doesn’t do anything to save space. If you have a million records with the key “somethingreallylongthatIshouldhaveshortened” you’re going to have a million instances of that string in your DB. There’s talk that Mongo might improve upon this, and Mongoid lets you name fields so that keys are stored as something smaller in the DB.