David Padilla of Crowd Interactive recently realized he was on this recruiting site called GitHire. They use algorithms to inspect your public GitHub repositories to determine how good of a programmer you are. Aside from the fact that this is a limited approach to "skill" which assumes that you use GitHub, I think that this tool has one strong intrinsic problem. This problem is worth talking about because I think that it is reflective of the whole "Startup Culture" I find myself in today.
With GitHub, you can follow folks, watch and fork repos, etc. GitHire makes the point to show followers and forks on profile pages, so I am assuming that those factor into their algorithm. The job of an engineer is to get shit done and make products / solve problems. The job of business development is to identify those problems, rank them meaningfully, get those products used (and presumably paid for). We live in strange times where those two skill sets are merging. Of course, I certainly fall into the description of "engineer who also does business development". Hell, I’m about to attempt another startup (more coming soon on that topic). I’m going to ignore my obvious hypocrisy for now, but I will address is later
When I think about hiring, I’m very tempted to fall into this trap. "Have you worked on any open source packages I have heard of?" or "Have you patched some important codebase?" are questions I need to force myself not to think. When hiring an engineer the right questions might be about "Scale", but they should remain technical in nature. Interview questions can remain about infrastructure and uptime in the context of userbase but not as a source of it. If 10K people downloaded your package from a sharing site you don’t maintain, that doesn’t tell me anything other than that you are really good at identifying engineering problems, but tells me nothing of your skill in the implementation. If 1K of those people who downloaded your package then forked it, that tells me you made a great start at solving the problem, but didn’t address the needs of (at least) 10% of your audience. You are a thought leader, you are a scientist. You might still be a terrible engineer.
There’s an important footnote to add here about framework / API development, which I believe might be an exception to the above. If your job is to create tools for other engineers to use, then adoption is a valuable measure of skill. The reason Rails is more popular than Merb has a lot to do with usability and distribution patterns. However, I don’t think that most engineers are hired with such a strong external focus. No engineer can work in a vacuum and other folks will need to interact with your code, but strong modularity is normally enough of an interaction and true APIs and DSLs aren’t needed.
Back to my Hypocrisy. When I’m applying for Engineering jobs, I find it best to narrow the scope of my accomplishments to just those of engineering. No engineering hiring manager really should care about how much money I saved the company as a Product Manager. They should care about how my architecture decisions allowed us to reduce the load on our servers, and scale down our hosting environment. See the difference?
TlDR; Good engineers are lazy. They automate code that writes code for them. In true unix fashion, good engineers are bricoleurs and their job is to build on the work of others, rather than reinventing the wheel each time. I don’t know how counting my lines of code and forks of my repo measures this.