Sep 11 2014

Using Data To Reduce Inequality

The racism we deal with today in America has evolved to be a lot more elusive than it was during the Civil Rights Movement. You can see its effects by looking at the country in aggregate. If you’re nonwhite, you’re more likely to be poor. You’re disproportionately more likely to have a run-in with the police, and when you do, you’re disproportionately more likely to be found guilty if you are prosecuted, and if you are found guilty, your sentences for the same crimes tend to be harsher than they would be if you were white.

It’s easy to see that in aggregate. You just have to measure it. But it’s more difficult to look at it on a case by case basis, because usually if there’s some racial bias happening, there’s plenty of plausible deniability surrounding it. The individual matched the description of someone who was reported to have robbed a convenience store is common in police reports, even when the individual is innocent.

Surprisingly, a technique that Tim Ferriss describes in his book The Four-Hour Body could be applicable here. In his book, he describes the importance of tracking data to lose weight. He found that merely using a scale to measure your weight every day leads to noticeable weight loss, even in the absence of any other diet or exercise regimen. Just seeing that data and being reminded of it every day is enough to affect tons of tiny, subsconscious decisions, and it made a noticeable difference.

What if we started using this technique with police officers? What if every police officer was briefed every week with the number of arrests they made, the racial breakdown of the arrests, and the racial breakdown of the local population. Using some statistics, this data could be tabulated into a score that indicates the likelihood that the officer is racially profiling civilians.

We don’t even need to give them quotas or targets to hit. We don’t need to have anyone breathing down officers’ necks about it. Hell, we don’t even need to share that data with the public. Just show it to each individual officer, as a constant reminder of the consequences of decisions they might not even be aware of. It’s so easy to think anecdotally about how fair you are, but when you see numbers that quantify it, it gets harder to rationalize.

I’m a really big fan of this technique for self-improvement, and it can be applied to any number of things. I work on a team of software engineers, and we could use data like this to help identify whether we have gender biases causing our lack of women on the team. You can use tools like RescueTime to know how much time you really spend checking your email during the work day.

On the surface, it’s just data. Just some numbers that happen to be correlated with happenings in the world. Being able to turn them into a better world is a beautiful thing.


Sep 10 2014

TIL

After being inspired by a bunch of Ruby Rogues episodes and in part by my colleague Brett’s year of Ruby tips I decided to try a simple experiment in which I get into the habit of starting to document every time I encounter some new piece of information. Sometimes days pass by when I feel like I haven’t grown in any appreciable way, and this is a really great and deliberate reminder that you learn something new every day, and you learn something new and remember to make a note of it almost every day.

Being the hardcore Evernote user that I am, I started dumping these notes into a new tag called things I learned today.

It’s a great habit. I find I remember these things a little better, and even when I don’t, it doesn’t matter! I have it saved safely in Evernote! I don’t just use this for programming topics, either. I make a note any time I figure out something and think “hey, that’s neat!” or if I was trying to figure out something and it was hard to google because I had such a tenuous grasp on how to describe it (like songs where the tune is stuck in my head but I can’t remember anything about the lyrics).

Here are a few things that I learned over the past several weeks:

Refinements in irb

It turns out that due to a change after refinements were introduced in Ruby 2.0.0, they’re not nearly as easy to use and play with in an irb console. You’re better off writing them in a file and including the file in IRB if you want to play. Learn more

Markdown to Evernote

There exists a handy OS X service for taking some Markdown and sending it into an Evernote note. I love this because I love to type bulleted lists using Markdown’s bulleted list syntax, but just having plain old Markdown in my Evernote note doesn’t feel nearly as fancy. The maintainer admits he has been neglecting it a bit, but you can check it out on his blog.

Regex Postgres queries with ActiveRecord

Here’s how to construct an ActiveRecord query to find records where a field matches a regex pattern:

things_with_a_pathetic_90s_description = Thing.where("description ~ '^.*NOT!$'")

A photographic diagram of how various ingredients affect the resulting cookie. Delicious…

Rendering collections without loops

Using ERB templates in Rails, there’s a concise way you can render a partial of every item in a collection:

<%= render partial:"line_item", collection:@line_items, as: :line_item %>

Enumerable#group_by

I love SQL’s GROUP BY. I wanted to do something similar with a plain old Ruby array, and in the process I learned why ActiveRecord calls it group and not group_by; the method name was taken. Also, that was the best I could do at coming up with a good mnemonic device for remembering which is for which.

(1..6).group_by { |i| i%3 } #=> {0=>[3, 6], 1=>[1, 4], 2=>[2, 5]}


Sep 7 2014

The Detroit Project

As many if you may know, I’m somewhat of an ideas guy. I’d like to share with you an idea of mine that’s been floating around.

I think I have a solution to the San Francisco’s ongoing issues. It’s overcrowded with technology startups. Unable to expand outward, rent and property prices are skyrocketing. Traffic is miserable. Long-time residents are being driven out of their homes. And whenever people try to develop new buildings to alleviate the issue, residents protest, both because these new buildings are favoring the rich, and because people don’t like the idea of a town they find so charming changing to a point where it’s no longer recognizable.

This is a human-made problem. None of these technology startups have a particular need for being on a coastal city; they’re technology companies, not shipping companies.

Meanwhile, America’s filled with ghost cities that suffer the very opposite issues. Their decreased occupancy leaves them with expensive infrastructure meant to support a larger population that the smaller population can no longer afford to support. Their economies are in the tank because incumbent industries abandoned them and the growing industries are looking to new darlings. With the mass exodus of people, housing prices are low to the point where you can buy homes for less than five figures.

Solution: Let’s turn Detroit into a desirable city for tech-driven companie, similar to San Francisco, Seattle, and Austin.

Here’s how:

We need to make the city desirable to trendy people who love tech startups. Detroit’s actually in a great position for this because if you look in any startup office (or, as they like to call it, a space), the ones that are considered nice don’t look like offices, but abandoned factories or warehouses. We just need to furnish them, and we’re done!

You need to move a lot of hipsters into Detroit as well. Hipsters are like trendiness canaries for coal mines, and if you send them in and they thrive, then you know the city will be considered hip. Plus the hipsters give the technology workers something to make fun of while they simultaneously emulate them. Hipsters also have a knack for thinking of trendy business and shop concepts that are quirky and would be successful in a big city, but non-hipsters are too sensible to ever think of them. Portland seems to have a surplus of hipsters, so we’ll bring some in from there.

Residential neighborhoods are another challenge. The ones with the cheapest housing are really dilapidated (and squatters are an issue). The secret weapon: gay couples. Get some neighborhoods with dilapidated housing but good nearby schools and bring in creative gay and lesbian couples. Before you know it, that neighborhood is desirable in no time.

Start buying up some buildings and start setting up incubators. Buy some blocks of houses, remodel and bring in entrepreneurs, telling them that in exchange for 10% of their equity, they don’t need to worry about essentials like housing or a workspace. Herman Miller is Michigan based so you should be able to truck in some trendy desk furniture and Eames chairs for a great price.

Being in Detroit’s going to give your startup a competitive edge: your housing and building costs are a fraction of what they are in SF, allowing you to bring in scrappy early employees for a lower price. If you’re an engineer living in Detroit working for an early stage startup for $65k a year you may as well be royalty. That $250k in seed funding wouldn’t cover your artisan coffee budget in SF, but you could have a small team working for a couple years on lots of great ideas if you’re in Detroit. Make the same investment, get twice the startups with twice the chances for success. Sounds like a win in my book.

You aren’t isolated from the world, either. Detroit’s airport is a Delta hub, connecting you to pretty much any major city. We’ll get Google to put in Google Fiber throughout the city (which will be really handy because everyone stripped the copper out of the empty homes to sell it as scrap metal). The population density is low enough that your 4G connection will actually perform pretty well.

And you know how SF is notorious for having a lot of homeless people with nowhere to go? Well, Detroit has a lot of empty houses with no one to live in them! With the ridiculously low building costs we can earmark some cash to create places to live for people who have suffered from a perpetually bad economy. And all this revitalizing is a lot of work, too… sounds like maybe there are some job opportunities on the horizon for the many people who are living in Detroit but need a break?

And on a more serious note, Detroit has something San Francisco is losing quickly: character and a sense of the plight of the working class American. In San Francisco, maybe it feels like the biggest problem you have to solve is that you need to interact with a human on the phone to get food delivered and you wish you could just get it with an app. If you put your startup in Detroit you’re going to have a perspective on life and your potential users San Francisco just can’t give you. I love a good delivery startup as much as the next guy, but I think there’s room for some more noble problems that are ready to be “disrupted.”

What do you say world? I’m an ideas guy, you’ll just have to run with this.


Sep 7 2014

ruby slots online casino

I’ve owned classic Apple mechanical keyboards for some time, but over the past few weeks I’ve been using the Apple Extended Keyboard and Apple Extended Keyboard II as my main keyboards and I really like them. They provide a lot of tactile feedback and they’re relatively quiet.

My biggest pet peeve with them has been the pesky fact that the Caps Lock key is actually a locking key. It achieves this by using a completely different Alps switch then the rest of the keyboard (the play 2 win online casino).

I am partial to swapping my Ctrl and Caps lock keys from their now-traditional positions because I use the Ctrl key all the time in keyboard shortcuts and I want it to be really accessible. Older Apple keyboards (like the https://icanthascheezburger.com/online-casino-247/) have those swapped already, but those keyboards have other idiosyncracies that are impractical for common use (such as the lack of function keys, and the Esc key being where the tilde key is, and the tilde key being by the really narrow spacebar). So, I set out to customize my best Apple Extended and Apple Extended II keyboards.

Swapping the Switches

Getting the caps lock key to stop locking down is a simple matter of replacing its Alps switch with a non-locking switch. In my case I decided to swap its switch with the left Ctrl key, which I’m perfectly content never to use.

Swapping mechanical switches is easy enough; it simply requires desoldering the pins from the PCB, popping them out, and resoldering them back in. I didn’t make a video of the process but I found a great can you win real money on cashman casino while browsing through resources on the MechanicalKeyboards star wars slot machine.

This was my first time soldering so I was a bit nervous, but mechanical keyboards are an easy enough project, as the solder points are relatively large and the Apple Extended Keyboard is famously easy to get apart (just four screws and the case comes right apart). Once I got the hang of desoldering (ProTip: even if you hold the solder wick a few inches from where you touch the solder iron to it, the heat travels really far!) the hardest part proved to be prying the switches out of their original homes. I felt like I was going to break the switches, but they were sturdy. I got the new ones in their homes (they just popped right in), and soldered them together.

not bad for my first time
not bad for my first time

Key remapping

Turns out, that was the easy part. It was my hypothesis that I would just need to swap the Ctrl and Caps Lock keys in System Preferences on OS X, but there was somewhat of a catch to it. To get the computer to treat the key as an actual toggle, the keyboard is hard wired to treat any key up or key down event as both a key down plus a key up event (to prevent the computer from seeing the key as held down for a long period of time). This was problematic for using the key as a Ctrl key, because Ctrl is a modifier key, so it needs to be held down with other keys, but as soon as you pressed it down it was treated as immediately released.

I needed to use a more low-level solution and I found one in Karabiner and Seil.

A helpful soul on GitHub had the same issue as I did and did the legwork of finding a solution: remapping the Caps lock key as a Ctrl lock key.

And it worked!

I have to do some further setup to Karabiner to make this only apply to specific keyboards (I don’t want the remapping to apply to my built-in keyboard, for that one I just want to do a simple Caps Lock/Ctrl swap which can be accomplished in System Preferences).

Karabiner looks like a promising app and it already appears to have support for OS X 10.10 (plus it’s open-sourced on GitHub) so this seems like it should be a good solution for the next few years or so.

I’m really happy with the keyboards I’m using as well. I was lucky enough to snag a new old stock Apple Extended II and it has a nice springy but dampened feel, so it’s really quiet and it was totally clean so the typing motion is perfect. My Apple Extended is used but in remarkably good condition for its age, and types great too. The spacebars on each take some getting used to for me, though. I think that the stabilizing motion that the Alps based keyboards use is more resistant than what’s used in keyboards using Cherry MX switches.

Another perk of using these Apple keyboards: the Command keys are correctly mapped and are more prominently placed on the keyboard for easier reach. They’re larger than the Alt keys on a Windows-formatted keyboard as well, which makes them all the more reachable.


Sep 6 2014

The Markdown Shitshow, Part Deux

So, after watching the Markdown standardization shitshow play out, some more substance is starting to come out of the woodwork.

First off, I was right in that people weren’t pissed about the name. Jeff Atwood changed the name to Common Markdown, then changed it again.

In the original renaming blog post, it was revealed that Gruber was emailed about the name Standard Markdown two weeks prior, and just never bothered to respond to that email. Instead, he approached the problem by publicly taking potshots at the project, and allegedly told Atwood that the name was “infuriating.” They tried another name, Common Markdown, but this was also a bust, and finally landed on CommonMark.

Case closed, right?

As of right now, Gruber’s still harboring ill will toward Atwood and others. It’s obvious that this isn’t about the name at all, but Gruber has yet to really articulate why; he seems to just be content to be an asshole about the project (and he invented it, so he certainly is entitled to be as big an asshole as he wants when people take his project and run with it, but I’m still calling him out on it).

Over the last couple of days, though, more context started to surface. Unfriendly exchanges between Atwood and Gruber date back many years, and there’s a great post indicating Gruber’s stance, which is essentially that he has a vendetta against standards.

That stance still wasn’t very satisfying to me. I figured there’s got to be more do it. Dr Drang posted a really solid case against what Commonmark is trying to do:

In other words, Standard Markdown isn’t a solidly built Core Markdown. Nor is it a Comprehensive Markdown with a bunch of helpful features that ol’ bastard Gruber refused to add. What it is is Yet Another Markdown Flavor, with a feature set tied to the needs of Meteor, Reddit, Stack Exchange, and GitHub. There’s nothing wrong with that, but it isn’t setting a Standard. It’s what everyone else does—some better, some worse. And in John MacFarlane’s case, he’s done it better at least three times.

Now we’re getting somewhere.

It would have been one thing if Commonmark took a hard stance and said “we’re just standardizing the core features of Markdown.” But they didn’t. They took that stance selectively, as evidenced by the GFM-inspired support for code fences, yet the lack of core support for things like tables. That seems hypocritical to me, and it’s a really valid complaint.

There are also objections to the fact that the spec was so full of really crazy edge cases and definitions of how to make the parser behave in those edge cases. After all, the spirit of Markdown is about the source being highly readable on its own pre-HTML conversion, and cases like this are are really fringe:

crazy edge case

And yes, it’s all well and good that the philosophy of Markdown is to not write weird markup, but if you want to write a robust parser for a language you don’t get the luxury of pretending edge and corner cases don’t exist. Gruber chose to write a parser that didn’t cover all these edge cases. As a Markdown user that’s not usually a big deal, but as someone who wants to write software that supports Markdown it starts to make things messy. What Commonmark created for programmers is a Markdown standard that unambiguously defines all possible input.

Here’s what I hope happens:

  • Commonmark should do one of two things: 1) drop the extras from Commonmark standard, or 2) incorporate the extras from other people’s Markdown forks to make an inclusive Markdown dialect. I personally would prefer the latter, assuming that the standard allows for a lot of flexibility on these extras. I personally love how Pandoc supports four ways to make tables
  • Gruber moves on and stops bringing it up, and either participates in the Commonmark community to help shape it, or follows his current strategy of being happy with what he initially created and sticking with that.
  • Commonmark becomes a de facto standard for Markdown formats, and when people are building their own Markdown forks, start off using Commonmark as their baseline, and extend from there.

Sep 4 2014

Standard Markdown

update: the name’s been changed to Common Markdown

Yesterday, Jeff Atwood announced that Standard Markdown, a project he had been working on for some time, was finally completed and ready for public review.

Like many a Markdown enthusiast, I have struggled with the fact that Markdown doesn’t have a standard. It does have an ambiguous definition which is a nice plain English way of describing how to use it, and it does have a reference implementation in Perl which people have leaned on as an official spec, but it too has ambiguities and bugs, and given that it hasn’t been touched in over ten years as of the time of this post’s writing, it’s safe to say that John Gruber (the co-creator of Markdown) has no interest in maintaining a spec for Markdown that is of the caliber needed for how widely it’s used, aside from a few fleeting mentions of his own preferences in some discussion boards.

I’m not upset that Gruber has left it untouched; I’m really happy that he and Aaron Swartz created it, but its situation has left us with some issues which Jeff Atwood has described in great detail, and Jeff Atwood set out to solve it by creating Standard Markdown.

The intent is misrepresented:

Jeff Atwood isn’t just going around being pissed that Markdown got abandoned from the top. He actually took the initiative to go about building a canonical standard, and he did in about as much the right way as a person can. He got together major players who are using Markdown in a big way (and often slight dialects of it), each with varying uses of Markdown (some using it for documentation like GitHub, others using it in forum software, etc.), and these companies worked together to build a solid standard of Markdown that everyone can work from, complete with a reference C and JavaScript implementation. There is now a detailed and unambiguous Markdown spec that sets rules on things that were previously unclear and deviates as little as possible from standard Markdown (and it doesn’t deviate from any of the major things you come to expect from any Markdown implementation).

Gruber’s reaction:

(this is @markdown’s second tweet in this account, the first is from several years ago describing Markdown)

A lot of people are taking the “I just hate that they called it that”stance:

That seems like an easy shot to take at Standard Markdown (easier than saying it was taken over, because Gruber has mostly stopped maintaining it). People are treating it as though this is some attempt to mislead people into thinking that Standard Markdown is the original Markdown.

Yes, the intent is absolutely there that Standard Markdown become the canonical Markdown, and it probably will become the de facto Markdown, because there’s so much weight behind it. The creators don’t see this as another fork of Markdown; we have plenty of well-defined forks and forks of forks (for instance: Pivotal Tracker made their own implementation of Markdown in the new Tracker, and described as a slight variant to GitHub Flavored Markdown).

Gruber is fully credited throughout Markdown’s spec, and great care was taken to make sure the spirit of Markdown is well-respected (after all, that’s what draws us to it). If you want to go and see Gruber’s spec in all its ambiguous glory, you can do so. There’s a link to it in the second paragraph of the home page of Standard Markdown’s site. Standard Markdown as a proper noun is a decent enough name. Already googling “standard markdown”will get you links to Atwood’s group’s implementation on the first page of results, and the name “standard markdown”is distinct from just plain old “Markdown”to indicate there is some standard to be followed.

I struggle to believe Gruber’s in all this fuss because of a name. Is Gruber upset he wasn’t consulted? He didn’t express interest in solidifying the spec when Atwood was lamenting the fragmentation of Markdown. If he had shown any interest in being involved in making a canonical unambiguous spec, it doesn’t seem like he would have been denied that opportunity. And even in the absence of his participation, his opinions were consulted in making the spec; there are links to his comments in there.

Is there worry that Markdown is being taken over by a consortium that might later grow to not represent users’ interests later on? I admit that worries me, but standards like Markdown shouldn’t evolve much (less so with a general-purpose canonical spec), and when they do, they should do so slowly and with discussion by stakeholders, exactly the type of thing a standards body solves. If new use cases emerge and Markdown isn’t adequate, people can adopt something new. And since Markdown is just plain text, no one’s ever really in danger of their data becoming inaccessible through Markdown falling out of favor.

Are people upset at specific design choices made about Standard Markdown? Are any of the changes breaking to the point where so-called Standard Markdown looks nothing like Markdown? None of the people in my Twitter feed mention that, and if they had, I’d consider those legitimate complaints, especially if this is to become a more widely adopted Markdown spec. And if they did have such concerns, that’s what the discussion site is for. But the people I’m seeing complain about Standard Markdown aren’t complaining about this.

Being upset that it’s named “Standard Markdown”is just petty. Standard Markdown is the result of a huge amount of work, and by taking a giant undertaking and redirecting all the attention onto its name it’s a slap in the face to the effort that people put into Standard Markdown in good faith to make it a better language. It feels like when Richard Stallman tried to insist that people refer to Linux as GNU/Linux. Maybe he was right and had the moral high ground, but it wasn’t a pragmatic choice.

John, if you’re reading this, Standard Markdown is the best thing to happen to your markup language since you invented it ten years ago. It’s too widely used for your original Markdown spec to be adequate for all use cases, and people stepped in and built a thorough spec and reference implementations. Something like this ensures the long-term health of Markdown, and what it needs above all is a thumbs-up from you, not quibbling over the name of the standard. I hope you review the standard and provide feedback and guidance on it. It would be sad if you became a footnote in Markdown’s history over something so minor as what a spec is named.