Saturday, October 11, 2008

Embarrassing Skills

No, this is not a post about shop-lifting or any other such antisocial skill. Besides, this is a blog about technology so expect me to refer to technical skills.

Logic would dictate that any skill remotely related to programming would be a feather in the hat of developers. After all, the more a developer knows the more valuable he/she could potentially be to a project, right? Unfortunately, many who became software developers after the birth of the web have decided that certain programming related skills are so shameful they will feign or pursue ignorance of them at all costs.

This avoidance of knowledge has let to some of the most inefficient and jumbled projects ever. But, before examining what kind of jumble can result from such attitude, let's name some names.

HTML
When presented with this acronym, many a developer will state the obvious: HTML is not a programming language. Bingo! It is not, neither is Postscript or Tex. Now it is kind of ridiculous that members of the generation that grew up making joke websites is so anti-HTML that they refuse to acknowledge the very existence of what has become the fastest growing display markup language. I am not leveraging this criticism of developers whose applications live on the desktop or on embedded systems. Such developers have their own user interface issues to worry about. If you write strictly Swing, MFC, Cocoa, or X11 applications and you do not know the first thing about HTML fine.

I am speaking of developers whose applications live on the web who cannot tell whether their applications are outputting the right kind of markup. That they would consider their ignorance (real or fake) to be a virtue, makes their attitude that more egregious. A J2EE or ASP .NET developer who does not know HTML is like a C++ developer who believes he/she is above learning about #defines or compiler switches.

JavaScript
This one is a favorite language to hate for many J2EE developers. The irony of it all is that, for better or for worse, Java is the closest relative to JavaScript. It's fair to say that JavaScript is as related to (or descended from) Java as Java is to C++. Yet, I have not met any C++ developers who disavow any knowledge or use of Java. For all of its flaws and poor structure, JavaScript is not likely to be replaceable as the web's premier scripting language in the same way that SQL will remain the premier database scripting language way into the future.

XML
This one is more an issue of prejudice since many developers don't seem to understand that, for the foreseeable future, there will always be need for heterogeneous systems to communicate over networks and that binary objects sent over multiple hops are too susceptible to corruption to be reliably transported. Also, in the case of XML, most developers that hate it just don't get it; the principle of serialization is lost on them.

Language
I mean language as in Human languages. The be more specific, written human languages. The truth about our profession is that we are creating systems that interact with humans in more complex ways. These systems will need to be more user-friendly at least and, in some cases, even human-like.

If you are a developer who thinks it is cool to not know stuff, remember that you will suffer from any technology you depend on professionally that you do not understand. Knowledge, my friend, is power — and, on rare occasions, it is also job security.

If you are a lead developer or project manager and you notice any of the above attitudes among your team, nip it in the bud because such behavior will eventually hurt your projects.

Tuesday, October 7, 2008

Be Careful of "Consultants"

As I was reading an article entitled saving energy in the data center on the InfoWorld site today, I was reminded of how IT managers can be misled by the IT Press and so-called consultants. Written by Logan G. Harbaugh, and Entitled 10 IT Power-Saving Myths Debunked, this article itself introduces at least one myth of its own. I am not alone in my assessment, as many of the readers proceeded to shred the author to pieces in their feedback comments.

Here for example is a real gem. Number two says, "It takes too long to cold-start servers to react to spikes in demand. If customers are made to wait, they'll go elsewhere." The author then proceeds to make asinine suggestions such as holding a site hostage until additional servers — which you turned off on his advice — are brought online. He supposes users will sympathize with your cause and wait around until your site is good to go.

Apart from providing some belly-holding, rolling-on-the-floor laughs, this bit of advice is not only naive, but it betrays absolute ignorance as to how data center and servers work.

First of all, Mr IT journalist guy, the users will go elsewhere — even card-carrying Green Party members seeking to buy recycled products will move on and will likely never come back — this is a lesson every e-commerce site owner learns real fast if he/she is to survive.

Secondly, have you any idea how long it takes for a server to boot up and make itself available on the network? Users expect their pages to load in 5 seconds or less.

Regardless of operating system, a data center server needs a good 5 minutes to boot up — and that is on a good day. This does not include integration into a server farm or registering with the load balancer. It is utterly impractical to even suggest that you can keep part of your server farm unplugged and bring it up in time to deal with increased traffic.

But the purpose of this post is not necessarily to beat up on Mr Harbaugh — although he does deserve the beating. Instead I want to focus on all the times and circumstances in which management listens to boneheaded advice from "consultants" or IT Journalists, as our friend Logan here, and hurt their businesses.

This happens so often that it poses a real threat to IT operations. I would include sales reps in this list, but I assume any MBA holder with half a brain is smart enough to realize that sales people will always say what their targets want to hear in order to get their respective commissions.

Consultants, consulting firms and trade publications (the poor manager's consultant) , however, are expected to be on the side of the people who come to them for advice. But it has often been the case that these people fail their trusting followers miserably.

If these failures were the product of human imperfections, I would not be so upset; but they are instead caused by other factors that should not be allowed to go unchecked, Here are a few along with — and you will love me for this — ways to recognize when you are being had.

1) Agendas and/or Causes
No matter how noble and just a cause, trade publications and consultants must reserve their first allegiance to you, their client or customer. When you go to the grocery store to buy milk, you do not expect to get pamphlet on animal cruelty instead of the gallon of the white stuff are there to pick up. You can tell preachy consultants by their lengthy speeches that have nothing to do with the task at hand. As terrible as the situation in Burma is, your server farm should be the topic of conversation in a meeting called for that purpose.

2)Unholy Alliances
Regardless of how good the products in question might be, a consultants or trade publications that take the side of vendors, are no longer what they claim to be. The have become mouthpieces and salespeople.
The job of a consultant (and, hopefully a professional journal) is to review products and services and only recommend the best of breed.
One easy way to tell these salespeople in disguise is by the way they give unqualified praise or criticism to products of a given brand, to the exclusion of all others in their class. This is especially evident when the consultant in question is ignorant of industry practices such as outsourced manufacturing and "badge engineering."

3)Impractical Advice/False Information
You normally trust consultants and IT journalists to provide you with information you do not readily have access to. It is only reasonable that you be able to trust the information they give you. A simple way to protect yourself here is to see what information you can get on facts you already know; if your consultant is wrong there, he/she is likely to be wrong in other areas as well.
Another great way to determine if you are dealing with the real McCoy is to watch out for defensiveness or unwillingness to answer questions you or your staff might have. Good consultants expect to be challenged; they welcome inquiries, are open to suggestions and listen a lot.
If your consultant is unresponsive and full of hubris, consider cutting your losses early. By the way, always protect your shop by placing clauses to deal with such things in your consulting contracts.

4)Information Hoarding and/or Secretiveness
Demand transparency from all you consultants. If a consultant is not transparent in his/her dealings, look elsewhere. Always stress the need for extensive documentation and knowledge transfer on behalf of your staff.
If a consultant insists on keeping you tethered without a really good reason — for example, you have just had a distributed supercomputer cluster deployed and its optimization will take a few months — take it as a clear sign that you should keep looking. This is not to say that you should limit your choices to open source solutions or that you should expect consultants to give away their trade secrets. Rather, what is advocated here is a reasonable delivery of a finished product with enough documentation and training to insure maximum enjoyment for the expected life of the product.


Don't allow yourself to be conned. No need to thank me. That's what I am here for.

Monday, October 6, 2008

Hack me once, shame on you; hack me twice, shame on me

I read a report about a Microsoft programming contest site that got hacked last week. I did a double take because I'd assumed Microsoft was past this sort of thing by now — what with their sites being the most targeted and all. But they keep getting hacked and they keep making excuses.

Although a big embarrassment to themselves, Microsoft is hardly alone in the ranks of the hacked. So instead of focusing on the Redmond giant alone, I want to dedicate this piece to all the shops who've had their sites hacked repeatedly and have taken no significant steps to stop the problem.

The truth is that it is possible to run bulletproof sites. I can confidently state that there is enough technology and expertise out there to stop 99% of attacks lobbed at a site and preempt the other 1%. I know this because I have been involved setting up secure sites and detecting potential attacks.

I know that, although there might be attacks that sneak through, a properly architectured site can handle even those that get past the first lines of defense.

The first thing to do in implementing security for an online site or application (or any application for that matter) is to make security part of the fundamental design of both software and hardware configuration.

It is indispensable that a security-oriented culture pervade all IT teams: development, deployment, network, hardware configuration/provisioning, architecture, project management, business analysts, etc. In the same way that we debug and refactor code, we need to make security a basic requirement.

The worse way to implement security is as an afterthought. This may take one of two forms. The most egregious case is when security is only mentioned into after a major breach. This usually leads to some directive to the effect that everyone should "secure" their applications, etc. Developers may add to their projects code that is claimed to fix the problem (which they copied from somewhere or another) and the problem is considered "solved"... until the next breach when the rinse cycle is repeated.

The other method is barely better. I call this the "someone found religion" way of dealing with security. This is when some project manager goes to a security conference, comes back all fired up, and a memo is sent to all developers to "please remember" to add security to their code — as if it were salt for their boiled eggs. Such memoes merely acknowledged before everyone gets back to business as usual.

The way to really tackle security is to deal with it at ever stage of development. From the very first moment that a business analyst hands off a set of requirements to the architect, security has to be the sine-qua-non of every subsequent stage. Every component or feature is to be planned and tested with security in mind.

Developers must be rewarded for finding and caulking security holes. Security needs to be included in every estimate and budget. Security must be the number one show-stopper in every release.

The application should not be signed off on until exhaustive security testing is done. Security is to be part of unit, regression and integration testing. Security should account for no less than 25% of all QA scripts and must be included in the UAT process as we;;.

With such a culture, embarrassing. money-losing breaches will become a thing of the past.

A lot of people might grumble that this position is not practical, that tight deadlines get in the way. Well, here is how you fit security into the real world, reengineer your operation until you can. Many shops have already done it, so there are plenty of examples to learn from.

Maybe the problem is release cycles or division of labor. Perhaps development methodologies are at fault. There could be a need to modify the technology mix or drop inefficient technologies. The truth is that, as our applications gain size and importance, we need to take security as seriously as automakers are required to.

Do you want to be responsible for the next theft of millions of credit card numbers or multimillion dollar losing e-commerce site crash? If you don't want to have to deal with the guilt and embarrassment which no amount of finger pointing will assuage, then you must make security a part of your development process today.

Wednesday, October 1, 2008

Does CAPTCHA Have a Future?

ZDNet and other tech sites are reporting that Microsoft's Hotmail CAPTCHA (Completely Automated Public Turing test to Tell Computers and Humans Apart) has been under attack again, even after the software giant made some changes to reduce the chances of spammers breaking through. The report says the CAPTCHA busting techniques used by the attackers have had a success rate as high as 15% — that's spammer heaven!

Incidents such as these have caused a lot of people to write off CAPTCHA. Unfortunately, unless a company wants to have humans pouring through questionable submissions, there really is no other alternative. Audio CAPTCHA (which is less an alternative than a complement), I am afraid, is actually easier to break than printed CAPTCHA.

I submit that the issue is creativity. Spammers and hackers have busted CAPTCHA more by grunt than by smarts. The odds are somewhat in their favor since they need not have a 100% success rate, while their victims have to be able to beat them every time.

Just like with authentication and credentials, CAPTCHA by itself is breakable with relative ease. However, this ease can be reduced by several orders of magnitude if we increase the number of factors that would generate a positive —remember two-factor authentication?

I have read that many spammers have actually used HAC (Human Assisted Computing) to break CAPTCHA. All the spammers have to do is setup a front site (offering things like movie downloads or pornography) and use the target CAPTCHA as if it were their own own. When the user of the front site passes the CAPTCHA test, the spammer immediately gains access to the target site.

This can be fought in many ways. Right off the bat, you probably want to limit or block image linking from external sites. If this is not possible, you could submit the URL of the form along with the user's CAPTCHA input. Any external site shenanigan will be detected with a simple URL match check.

Beyond getting more from CAPTCHA as it exists today, I would suggest improving on the current technology. For example, a photograph accompanied by questions only a human could answer, such as "which person in this picture appears to be youngest?" Another idea would be to use a short movie and ask the person to describe what is going on, or what action preceded what other action.

Another idea would be to analyze the users behavior to determine if it is a human or a machine. Following mouse movements for as little as 1 second can tell whether a hand is human or a simulated. Most scripts do not produce any mouse movement and if they do it is likely to be stilted.

Another thing technologists and developers should remember is that CAPTCHA is more a concept than a technology — like artificial intelligence. As long as the objective is achieved we can call whatever we do CAPTCHA.

As computers become more sophisticated, we will have to come up with new Shibboleths to entrap non-human users. All it will take is us staying in touch with our creative sides.