This being the holiday season and most of you doing your hardest not to think of work -- unless you are on call in which case nothing I write here can top whatever the pager on your hip says I will deal with a relatively light topic but still one of relevance: server names.
In the early days of the Internet say, in the late 80s and early 90s when most servers were located on university campuses they were usually named after cartoon characters such as Snoopy.
Larger businesses preferred geographic names and used them in conjunction with departmental naming. Thus a server that was primarily used by accounting in Denver would be named something like denver_accounting; if they got a second server it would be named denver_accounting2 and so on.
These schemes appeared to work for a while, but increases in the number of physical servers combined with the surge in virtualization not to mention the rise of mixed use servers or cross-department applications has all but exhausted current naming schemes.
The best proof of this are the bizarre names popping up in current IT server farms. For example, some administrators just use the default name suggested by the OS on install. This results in names that are meaningless and difficult to remember such as GHXF13M or some such.
There are many articles out there that give all kinds of reasons why a server's name should reflect purpose. However, given the rate of change in modern organizations this would mean changing names for servers fairly often which would add to the network administrators list of tasks unnecessarily. The only time such a naming convention makes sense would be if the server in question is virtual and it has a specialized use. Servers that have mixed use are better off having memorable names that fit within a scalable scheme.
Another thing to remember is that, given the litigiousness of our society the the naming scheme needs to be neutral enough to not offend most people. An online article I read, described how administrators at a certain company would name their new servers based on how they felt after about the food served on a given day at their company cafeteria; so they ended up with names like "Squishy". Apart from the limited scalability of such a scheme, the potential for litigation is big enough to make it unsuitable.
So far we have fleshed out 3 principles for creating a server naming convention:
(1) User-friendliness or memorable names
(2) Scalability or a long enough list
(3) Reasonably non-offensive or prone to litigation
What remains now is to examine how we can implement each in practical terms.
1. User-friendly names
The most common non-human names we use on a daily basis are street names. The most easily remembered yet common street names correspond to names of individuals or places. The Washington, DC, is famous for using names of states on many of their streets. Names of insects might also fly, pun intended, except perhaps for some of the more queasiness-inducing ones.
2. Scalability
The main problem with rule number two is that it is in direct conflict with rule number 1. Obviously, the use of US state and territory names is not a good idea because there are less than 100. County names might be a good idea since they number over 3,000. However, no matter how big the name set used, it will have a finite limit and there ought to be a plan B for when the first set is exhausted. You should probably have on hand at least three sets of names to be used in turn as the preceding one becomes exhausted.
3. Reasonably non-offensive
Before I identify sources of names, I would like to suggest that such sources be documented for two reasons :
(a) as guide to your successors
(b) as a counter-argument to anyone who might deem a name as offensive
Now, lets turn to the sources of name sets. The US Census bureau has some wonderful data files that one can download from their web site and use free of charge or legal constraint . I especially recommend the Gazetteer section. A quick glance shows the following data sets with perfectly extractable names:
Counties (3,141 records)
MCDs [Minor Civil Divisions] (36,289 records)
Places (23,789 records)
Zips (29,470 records)
Online phone books are useful as long as you only use last names, and so are baby name sites, although I would recommend staying away from the more common ones. Even sites not usually associated with lists, such as Wikipedia, can be sources of lists that are long enough and have names that are easy enough to remember.
Now with all these names to administer you are going to need a good database to make sure there are no mixups. LDAP and/or Active Directory or whatever equivalent you have at your firm are good for basic management and search. However, for better navigability and even to store unassigned names for easy accessibility and quick implementation, an external database with an easy to use web interface would be best. Ideally the server management software should keep both databases (LDAP/AD an the server list database) synchronized and allow for search based on any combination of criteria.
In the end this should make the management of your server farms easier at a time when server proliferation is such that a company need not be named Google to list thousands of servers in its farm or farms.
I would go on to speak of workstation names, but they are of less concern. Besides, there is software out there that can do that almost automatically (such as the Windows 2000 Remote Installation Service). However, if you still need ideas for workstation names I would point you to the baby names sites for an easy source.
but trends that I have noticed suggest to me that we need a more comprehensive and scalable naming conventions than have previously been used
Some of the larger corporations have a way of
Notes ,observations and advice on software development/architecture and technology in general.
Thursday, December 25, 2008
Monday, December 8, 2008
Patterns and Other Buzzwords
Recently I was reading another blog post by way of Slashdot about MVC which reminded me of a pet peeve I think is worth bringing up.
Let me begin by saying that architectural patterns like MVC or software design patterns like Singleton are a wonderful way to encapsulate high or mid-level software structures for easy re-use or combination.
Like plays in an NFL team's playbook, they provide the software developer with prepackaged strategies or tools to tackle a particular problem. Therefore I can only recommend that developers learn all they can about as many patterns as they can and stretch their brains figuring out under what conditions each could best be utilized.
What is sickening, if not disheartening, is to see such great concepts reduced to buzzwords. Too many developers plan their projects with merely the buzzwords in mind damned be reliability, adequacy, security, scalability or the customer/user's needs. Its all about saying that they got to use the Facade pattern combined with the Abstract Factory along with the Observer, the Visitor and the Active Object patterns and ... oh, it was all very MVC.
This is putting the cart ahead of the horse. The right way to do it is to begin by assessing what needs to be done and then choosing the right tools for the job.
In the same way that we are careful about hardware provisioning and platform, IDE, and language choice, we should plan our choice or patterns to match the desired result. This might mean that mid-project we may have to dump some patterns chosen earlier and enlist others. It may mean that we may have to increase the complexity of the project or simplify it usually the latter. The point is the finished product should be the focus of all our efforts. The cult of buzzwords can only hurt the quality of our work.
And before I go, I should remind everyone that no amount of buzzwords can replace well-written code and proper documentation. These two cannot be emphasized enough. Many developers enjoy indulging in obfuscation without realizing that they are making it easier for the next guy to blame them even for new bugs introduced into their murky code.
So, do yourself a favor, forget the buzzwords. Choose the patterns that suit your project or invent your own if there is none that can fit hard to imagine but the existing patterns had to be invented at some point. Refactor as much as you can think of refactoring as preemptive debugging (if often is). Document as much as you can, either through a tool or by means of helpful comments in code; often an older you will be your most avid reader.
And finally, learn as much as you can about design patterns, especially their strengths and weaknesses. Patterns are very powerful if used judiciously, but disastrous when used carelessly. Now go change the world with your code.
Let me begin by saying that architectural patterns like MVC or software design patterns like Singleton are a wonderful way to encapsulate high or mid-level software structures for easy re-use or combination.
Like plays in an NFL team's playbook, they provide the software developer with prepackaged strategies or tools to tackle a particular problem. Therefore I can only recommend that developers learn all they can about as many patterns as they can and stretch their brains figuring out under what conditions each could best be utilized.
What is sickening, if not disheartening, is to see such great concepts reduced to buzzwords. Too many developers plan their projects with merely the buzzwords in mind damned be reliability, adequacy, security, scalability or the customer/user's needs. Its all about saying that they got to use the Facade pattern combined with the Abstract Factory along with the Observer, the Visitor and the Active Object patterns and ... oh, it was all very MVC.
This is putting the cart ahead of the horse. The right way to do it is to begin by assessing what needs to be done and then choosing the right tools for the job.
In the same way that we are careful about hardware provisioning and platform, IDE, and language choice, we should plan our choice or patterns to match the desired result. This might mean that mid-project we may have to dump some patterns chosen earlier and enlist others. It may mean that we may have to increase the complexity of the project or simplify it usually the latter. The point is the finished product should be the focus of all our efforts. The cult of buzzwords can only hurt the quality of our work.
And before I go, I should remind everyone that no amount of buzzwords can replace well-written code and proper documentation. These two cannot be emphasized enough. Many developers enjoy indulging in obfuscation without realizing that they are making it easier for the next guy to blame them even for new bugs introduced into their murky code.
So, do yourself a favor, forget the buzzwords. Choose the patterns that suit your project or invent your own if there is none that can fit hard to imagine but the existing patterns had to be invented at some point. Refactor as much as you can think of refactoring as preemptive debugging (if often is). Document as much as you can, either through a tool or by means of helpful comments in code; often an older you will be your most avid reader.
And finally, learn as much as you can about design patterns, especially their strengths and weaknesses. Patterns are very powerful if used judiciously, but disastrous when used carelessly. Now go change the world with your code.
Thursday, November 20, 2008
Misleading Figures
Many a project manager is pressed to, or concerned about, measuring developer productivity and some reach for the lowest hanging fruit : lines of code. It is unbelievable he kind of things that still go on some IT shops!
This is not only an imprecise metric in encourages code bloat and other bad programming practices while discouraging mainstays of good programming such as debugging and refactoring.
Some who agree with my points above may be devotees of another less metric that is slightly less misleading: the number of builds or check-ins. While it is true that a productive developer is likely to check in and build code often, an unproductive developer can easily game this system to appear most productive.
Yet another way of measuring productivity is by means of bug report or enhancement request tickets. This is probably the most realistic method of gathering developer productivity metrics, but it can also be misleading. In some cases the incident response system is also used for billing a client. This does not necessarily translate as tempting as it might be.
So, how do you reliably and fairly measure developer productivity? You don't. To put it a better way, you shouldn't have to. A bug tracking/change request tool is very useful but it should be used the way sports teams used game videos to learn from past experiences and improve performance.
Hold on to that sports analogy because that is a good place to be. A development shop is a team effort and what makes a developer valuable is how much a contribution he/she makes. Although different metrics may point to the top performers, a good project manager needs to be wise enough to know the strengths and weaknesses of each team member and position him/her for maximum impact.
The productivity measured and relied on most should be that of the team. This is not to say that we should give cover to freeloaders and underperformers; these should be dealt with as soon and as severely as possible. This means that sometimes more can be achieved as we realize that software development is more like a barn raising than a marathon.
This is not only an imprecise metric in encourages code bloat and other bad programming practices while discouraging mainstays of good programming such as debugging and refactoring.
Some who agree with my points above may be devotees of another less metric that is slightly less misleading: the number of builds or check-ins. While it is true that a productive developer is likely to check in and build code often, an unproductive developer can easily game this system to appear most productive.
Yet another way of measuring productivity is by means of bug report or enhancement request tickets. This is probably the most realistic method of gathering developer productivity metrics, but it can also be misleading. In some cases the incident response system is also used for billing a client. This does not necessarily translate as tempting as it might be.
So, how do you reliably and fairly measure developer productivity? You don't. To put it a better way, you shouldn't have to. A bug tracking/change request tool is very useful but it should be used the way sports teams used game videos to learn from past experiences and improve performance.
Hold on to that sports analogy because that is a good place to be. A development shop is a team effort and what makes a developer valuable is how much a contribution he/she makes. Although different metrics may point to the top performers, a good project manager needs to be wise enough to know the strengths and weaknesses of each team member and position him/her for maximum impact.
The productivity measured and relied on most should be that of the team. This is not to say that we should give cover to freeloaders and underperformers; these should be dealt with as soon and as severely as possible. This means that sometimes more can be achieved as we realize that software development is more like a barn raising than a marathon.
Saturday, October 11, 2008
Embarrassing Skills
No, this is not a post about shop-lifting or any other such antisocial skill. Besides, this is a blog about technology so expect me to refer to technical skills.
Logic would dictate that any skill remotely related to programming would be a feather in the hat of developers. After all, the more a developer knows the more valuable he/she could potentially be to a project, right? Unfortunately, many who became software developers after the birth of the web have decided that certain programming related skills are so shameful they will feign or pursue ignorance of them at all costs.
This avoidance of knowledge has let to some of the most inefficient and jumbled projects ever. But, before examining what kind of jumble can result from such attitude, let's name some names.
HTML
When presented with this acronym, many a developer will state the obvious: HTML is not a programming language. Bingo! It is not, neither is Postscript or Tex. Now it is kind of ridiculous that members of the generation that grew up making joke websites is so anti-HTML that they refuse to acknowledge the very existence of what has become the fastest growing display markup language. I am not leveraging this criticism of developers whose applications live on the desktop or on embedded systems. Such developers have their own user interface issues to worry about. If you write strictly Swing, MFC, Cocoa, or X11 applications and you do not know the first thing about HTML fine.
I am speaking of developers whose applications live on the web who cannot tell whether their applications are outputting the right kind of markup. That they would consider their ignorance (real or fake) to be a virtue, makes their attitude that more egregious. A J2EE or ASP .NET developer who does not know HTML is like a C++ developer who believes he/she is above learning about #defines or compiler switches.
JavaScript
This one is a favorite language to hate for many J2EE developers. The irony of it all is that, for better or for worse, Java is the closest relative to JavaScript. It's fair to say that JavaScript is as related to (or descended from) Java as Java is to C++. Yet, I have not met any C++ developers who disavow any knowledge or use of Java. For all of its flaws and poor structure, JavaScript is not likely to be replaceable as the web's premier scripting language in the same way that SQL will remain the premier database scripting language way into the future.
XML
This one is more an issue of prejudice since many developers don't seem to understand that, for the foreseeable future, there will always be need for heterogeneous systems to communicate over networks and that binary objects sent over multiple hops are too susceptible to corruption to be reliably transported. Also, in the case of XML, most developers that hate it just don't get it; the principle of serialization is lost on them.
Language
I mean language as in Human languages. The be more specific, written human languages. The truth about our profession is that we are creating systems that interact with humans in more complex ways. These systems will need to be more user-friendly at least and, in some cases, even human-like.
If you are a developer who thinks it is cool to not know stuff, remember that you will suffer from any technology you depend on professionally that you do not understand. Knowledge, my friend, is power and, on rare occasions, it is also job security.
If you are a lead developer or project manager and you notice any of the above attitudes among your team, nip it in the bud because such behavior will eventually hurt your projects.
Logic would dictate that any skill remotely related to programming would be a feather in the hat of developers. After all, the more a developer knows the more valuable he/she could potentially be to a project, right? Unfortunately, many who became software developers after the birth of the web have decided that certain programming related skills are so shameful they will feign or pursue ignorance of them at all costs.
This avoidance of knowledge has let to some of the most inefficient and jumbled projects ever. But, before examining what kind of jumble can result from such attitude, let's name some names.
HTML
When presented with this acronym, many a developer will state the obvious: HTML is not a programming language. Bingo! It is not, neither is Postscript or Tex. Now it is kind of ridiculous that members of the generation that grew up making joke websites is so anti-HTML that they refuse to acknowledge the very existence of what has become the fastest growing display markup language. I am not leveraging this criticism of developers whose applications live on the desktop or on embedded systems. Such developers have their own user interface issues to worry about. If you write strictly Swing, MFC, Cocoa, or X11 applications and you do not know the first thing about HTML fine.
I am speaking of developers whose applications live on the web who cannot tell whether their applications are outputting the right kind of markup. That they would consider their ignorance (real or fake) to be a virtue, makes their attitude that more egregious. A J2EE or ASP .NET developer who does not know HTML is like a C++ developer who believes he/she is above learning about #defines or compiler switches.
JavaScript
This one is a favorite language to hate for many J2EE developers. The irony of it all is that, for better or for worse, Java is the closest relative to JavaScript. It's fair to say that JavaScript is as related to (or descended from) Java as Java is to C++. Yet, I have not met any C++ developers who disavow any knowledge or use of Java. For all of its flaws and poor structure, JavaScript is not likely to be replaceable as the web's premier scripting language in the same way that SQL will remain the premier database scripting language way into the future.
XML
This one is more an issue of prejudice since many developers don't seem to understand that, for the foreseeable future, there will always be need for heterogeneous systems to communicate over networks and that binary objects sent over multiple hops are too susceptible to corruption to be reliably transported. Also, in the case of XML, most developers that hate it just don't get it; the principle of serialization is lost on them.
Language
I mean language as in Human languages. The be more specific, written human languages. The truth about our profession is that we are creating systems that interact with humans in more complex ways. These systems will need to be more user-friendly at least and, in some cases, even human-like.
If you are a developer who thinks it is cool to not know stuff, remember that you will suffer from any technology you depend on professionally that you do not understand. Knowledge, my friend, is power and, on rare occasions, it is also job security.
If you are a lead developer or project manager and you notice any of the above attitudes among your team, nip it in the bud because such behavior will eventually hurt your projects.
Tuesday, October 7, 2008
Be Careful of "Consultants"
As I was reading an article entitled saving energy in the data center on the InfoWorld site today, I was reminded of how IT managers can be misled by the IT Press and so-called consultants. Written by Logan G. Harbaugh, and Entitled 10 IT Power-Saving Myths Debunked, this article itself introduces at least one myth of its own. I am not alone in my assessment, as many of the readers proceeded to shred the author to pieces in their feedback comments.
Here for example is a real gem. Number two says, "It takes too long to cold-start servers to react to spikes in demand. If customers are made to wait, they'll go elsewhere." The author then proceeds to make asinine suggestions such as holding a site hostage until additional servers which you turned off on his advice are brought online. He supposes users will sympathize with your cause and wait around until your site is good to go.
Apart from providing some belly-holding, rolling-on-the-floor laughs, this bit of advice is not only naive, but it betrays absolute ignorance as to how data center and servers work.
First of all, Mr IT journalist guy, the users will go elsewhere even card-carrying Green Party members seeking to buy recycled products will move on and will likely never come back this is a lesson every e-commerce site owner learns real fast if he/she is to survive.
Secondly, have you any idea how long it takes for a server to boot up and make itself available on the network? Users expect their pages to load in 5 seconds or less.
Regardless of operating system, a data center server needs a good 5 minutes to boot up and that is on a good day. This does not include integration into a server farm or registering with the load balancer. It is utterly impractical to even suggest that you can keep part of your server farm unplugged and bring it up in time to deal with increased traffic.
But the purpose of this post is not necessarily to beat up on Mr Harbaugh although he does deserve the beating. Instead I want to focus on all the times and circumstances in which management listens to boneheaded advice from "consultants" or IT Journalists, as our friend Logan here, and hurt their businesses.
This happens so often that it poses a real threat to IT operations. I would include sales reps in this list, but I assume any MBA holder with half a brain is smart enough to realize that sales people will always say what their targets want to hear in order to get their respective commissions.
Consultants, consulting firms and trade publications (the poor manager's consultant) , however, are expected to be on the side of the people who come to them for advice. But it has often been the case that these people fail their trusting followers miserably.
If these failures were the product of human imperfections, I would not be so upset; but they are instead caused by other factors that should not be allowed to go unchecked, Here are a few along with and you will love me for this ways to recognize when you are being had.
1) Agendas and/or Causes
No matter how noble and just a cause, trade publications and consultants must reserve their first allegiance to you, their client or customer. When you go to the grocery store to buy milk, you do not expect to get pamphlet on animal cruelty instead of the gallon of the white stuff are there to pick up. You can tell preachy consultants by their lengthy speeches that have nothing to do with the task at hand. As terrible as the situation in Burma is, your server farm should be the topic of conversation in a meeting called for that purpose.
2)Unholy Alliances
Regardless of how good the products in question might be, a consultants or trade publications that take the side of vendors, are no longer what they claim to be. The have become mouthpieces and salespeople.
The job of a consultant (and, hopefully a professional journal) is to review products and services and only recommend the best of breed.
One easy way to tell these salespeople in disguise is by the way they give unqualified praise or criticism to products of a given brand, to the exclusion of all others in their class. This is especially evident when the consultant in question is ignorant of industry practices such as outsourced manufacturing and "badge engineering."
3)Impractical Advice/False Information
You normally trust consultants and IT journalists to provide you with information you do not readily have access to. It is only reasonable that you be able to trust the information they give you. A simple way to protect yourself here is to see what information you can get on facts you already know; if your consultant is wrong there, he/she is likely to be wrong in other areas as well.
Another great way to determine if you are dealing with the real McCoy is to watch out for defensiveness or unwillingness to answer questions you or your staff might have. Good consultants expect to be challenged; they welcome inquiries, are open to suggestions and listen a lot.
If your consultant is unresponsive and full of hubris, consider cutting your losses early. By the way, always protect your shop by placing clauses to deal with such things in your consulting contracts.
4)Information Hoarding and/or Secretiveness
Demand transparency from all you consultants. If a consultant is not transparent in his/her dealings, look elsewhere. Always stress the need for extensive documentation and knowledge transfer on behalf of your staff.
If a consultant insists on keeping you tethered without a really good reason for example, you have just had a distributed supercomputer cluster deployed and its optimization will take a few months take it as a clear sign that you should keep looking. This is not to say that you should limit your choices to open source solutions or that you should expect consultants to give away their trade secrets. Rather, what is advocated here is a reasonable delivery of a finished product with enough documentation and training to insure maximum enjoyment for the expected life of the product.
Don't allow yourself to be conned. No need to thank me. That's what I am here for.
Here for example is a real gem. Number two says, "It takes too long to cold-start servers to react to spikes in demand. If customers are made to wait, they'll go elsewhere." The author then proceeds to make asinine suggestions such as holding a site hostage until additional servers which you turned off on his advice are brought online. He supposes users will sympathize with your cause and wait around until your site is good to go.
Apart from providing some belly-holding, rolling-on-the-floor laughs, this bit of advice is not only naive, but it betrays absolute ignorance as to how data center and servers work.
First of all, Mr IT journalist guy, the users will go elsewhere even card-carrying Green Party members seeking to buy recycled products will move on and will likely never come back this is a lesson every e-commerce site owner learns real fast if he/she is to survive.
Secondly, have you any idea how long it takes for a server to boot up and make itself available on the network? Users expect their pages to load in 5 seconds or less.
Regardless of operating system, a data center server needs a good 5 minutes to boot up and that is on a good day. This does not include integration into a server farm or registering with the load balancer. It is utterly impractical to even suggest that you can keep part of your server farm unplugged and bring it up in time to deal with increased traffic.
But the purpose of this post is not necessarily to beat up on Mr Harbaugh although he does deserve the beating. Instead I want to focus on all the times and circumstances in which management listens to boneheaded advice from "consultants" or IT Journalists, as our friend Logan here, and hurt their businesses.
This happens so often that it poses a real threat to IT operations. I would include sales reps in this list, but I assume any MBA holder with half a brain is smart enough to realize that sales people will always say what their targets want to hear in order to get their respective commissions.
Consultants, consulting firms and trade publications (the poor manager's consultant) , however, are expected to be on the side of the people who come to them for advice. But it has often been the case that these people fail their trusting followers miserably.
If these failures were the product of human imperfections, I would not be so upset; but they are instead caused by other factors that should not be allowed to go unchecked, Here are a few along with and you will love me for this ways to recognize when you are being had.
1) Agendas and/or Causes
No matter how noble and just a cause, trade publications and consultants must reserve their first allegiance to you, their client or customer. When you go to the grocery store to buy milk, you do not expect to get pamphlet on animal cruelty instead of the gallon of the white stuff are there to pick up. You can tell preachy consultants by their lengthy speeches that have nothing to do with the task at hand. As terrible as the situation in Burma is, your server farm should be the topic of conversation in a meeting called for that purpose.
2)Unholy Alliances
Regardless of how good the products in question might be, a consultants or trade publications that take the side of vendors, are no longer what they claim to be. The have become mouthpieces and salespeople.
The job of a consultant (and, hopefully a professional journal) is to review products and services and only recommend the best of breed.
One easy way to tell these salespeople in disguise is by the way they give unqualified praise or criticism to products of a given brand, to the exclusion of all others in their class. This is especially evident when the consultant in question is ignorant of industry practices such as outsourced manufacturing and "badge engineering."
3)Impractical Advice/False Information
You normally trust consultants and IT journalists to provide you with information you do not readily have access to. It is only reasonable that you be able to trust the information they give you. A simple way to protect yourself here is to see what information you can get on facts you already know; if your consultant is wrong there, he/she is likely to be wrong in other areas as well.
Another great way to determine if you are dealing with the real McCoy is to watch out for defensiveness or unwillingness to answer questions you or your staff might have. Good consultants expect to be challenged; they welcome inquiries, are open to suggestions and listen a lot.
If your consultant is unresponsive and full of hubris, consider cutting your losses early. By the way, always protect your shop by placing clauses to deal with such things in your consulting contracts.
4)Information Hoarding and/or Secretiveness
Demand transparency from all you consultants. If a consultant is not transparent in his/her dealings, look elsewhere. Always stress the need for extensive documentation and knowledge transfer on behalf of your staff.
If a consultant insists on keeping you tethered without a really good reason for example, you have just had a distributed supercomputer cluster deployed and its optimization will take a few months take it as a clear sign that you should keep looking. This is not to say that you should limit your choices to open source solutions or that you should expect consultants to give away their trade secrets. Rather, what is advocated here is a reasonable delivery of a finished product with enough documentation and training to insure maximum enjoyment for the expected life of the product.
Don't allow yourself to be conned. No need to thank me. That's what I am here for.
Monday, October 6, 2008
Hack me once, shame on you; hack me twice, shame on me
I read a report about a Microsoft programming contest site that got hacked last week. I did a double take because I'd assumed Microsoft was past this sort of thing by now what with their sites being the most targeted and all. But they keep getting hacked and they keep making excuses.
Although a big embarrassment to themselves, Microsoft is hardly alone in the ranks of the hacked. So instead of focusing on the Redmond giant alone, I want to dedicate this piece to all the shops who've had their sites hacked repeatedly and have taken no significant steps to stop the problem.
The truth is that it is possible to run bulletproof sites. I can confidently state that there is enough technology and expertise out there to stop 99% of attacks lobbed at a site and preempt the other 1%. I know this because I have been involved setting up secure sites and detecting potential attacks.
I know that, although there might be attacks that sneak through, a properly architectured site can handle even those that get past the first lines of defense.
The first thing to do in implementing security for an online site or application (or any application for that matter) is to make security part of the fundamental design of both software and hardware configuration.
It is indispensable that a security-oriented culture pervade all IT teams: development, deployment, network, hardware configuration/provisioning, architecture, project management, business analysts, etc. In the same way that we debug and refactor code, we need to make security a basic requirement.
The worse way to implement security is as an afterthought. This may take one of two forms. The most egregious case is when security is only mentioned into after a major breach. This usually leads to some directive to the effect that everyone should "secure" their applications, etc. Developers may add to their projects code that is claimed to fix the problem (which they copied from somewhere or another) and the problem is considered "solved"... until the next breach when the rinse cycle is repeated.
The other method is barely better. I call this the "someone found religion" way of dealing with security. This is when some project manager goes to a security conference, comes back all fired up, and a memo is sent to all developers to "please remember" to add security to their code as if it were salt for their boiled eggs. Such memoes merely acknowledged before everyone gets back to business as usual.
The way to really tackle security is to deal with it at ever stage of development. From the very first moment that a business analyst hands off a set of requirements to the architect, security has to be the sine-qua-non of every subsequent stage. Every component or feature is to be planned and tested with security in mind.
Developers must be rewarded for finding and caulking security holes. Security needs to be included in every estimate and budget. Security must be the number one show-stopper in every release.
The application should not be signed off on until exhaustive security testing is done. Security is to be part of unit, regression and integration testing. Security should account for no less than 25% of all QA scripts and must be included in the UAT process as we;;.
With such a culture, embarrassing. money-losing breaches will become a thing of the past.
A lot of people might grumble that this position is not practical, that tight deadlines get in the way. Well, here is how you fit security into the real world, reengineer your operation until you can. Many shops have already done it, so there are plenty of examples to learn from.
Maybe the problem is release cycles or division of labor. Perhaps development methodologies are at fault. There could be a need to modify the technology mix or drop inefficient technologies. The truth is that, as our applications gain size and importance, we need to take security as seriously as automakers are required to.
Do you want to be responsible for the next theft of millions of credit card numbers or multimillion dollar losing e-commerce site crash? If you don't want to have to deal with the guilt and embarrassment which no amount of finger pointing will assuage, then you must make security a part of your development process today.
Although a big embarrassment to themselves, Microsoft is hardly alone in the ranks of the hacked. So instead of focusing on the Redmond giant alone, I want to dedicate this piece to all the shops who've had their sites hacked repeatedly and have taken no significant steps to stop the problem.
The truth is that it is possible to run bulletproof sites. I can confidently state that there is enough technology and expertise out there to stop 99% of attacks lobbed at a site and preempt the other 1%. I know this because I have been involved setting up secure sites and detecting potential attacks.
I know that, although there might be attacks that sneak through, a properly architectured site can handle even those that get past the first lines of defense.
The first thing to do in implementing security for an online site or application (or any application for that matter) is to make security part of the fundamental design of both software and hardware configuration.
It is indispensable that a security-oriented culture pervade all IT teams: development, deployment, network, hardware configuration/provisioning, architecture, project management, business analysts, etc. In the same way that we debug and refactor code, we need to make security a basic requirement.
The worse way to implement security is as an afterthought. This may take one of two forms. The most egregious case is when security is only mentioned into after a major breach. This usually leads to some directive to the effect that everyone should "secure" their applications, etc. Developers may add to their projects code that is claimed to fix the problem (which they copied from somewhere or another) and the problem is considered "solved"... until the next breach when the rinse cycle is repeated.
The other method is barely better. I call this the "someone found religion" way of dealing with security. This is when some project manager goes to a security conference, comes back all fired up, and a memo is sent to all developers to "please remember" to add security to their code as if it were salt for their boiled eggs. Such memoes merely acknowledged before everyone gets back to business as usual.
The way to really tackle security is to deal with it at ever stage of development. From the very first moment that a business analyst hands off a set of requirements to the architect, security has to be the sine-qua-non of every subsequent stage. Every component or feature is to be planned and tested with security in mind.
Developers must be rewarded for finding and caulking security holes. Security needs to be included in every estimate and budget. Security must be the number one show-stopper in every release.
The application should not be signed off on until exhaustive security testing is done. Security is to be part of unit, regression and integration testing. Security should account for no less than 25% of all QA scripts and must be included in the UAT process as we;;.
With such a culture, embarrassing. money-losing breaches will become a thing of the past.
A lot of people might grumble that this position is not practical, that tight deadlines get in the way. Well, here is how you fit security into the real world, reengineer your operation until you can. Many shops have already done it, so there are plenty of examples to learn from.
Maybe the problem is release cycles or division of labor. Perhaps development methodologies are at fault. There could be a need to modify the technology mix or drop inefficient technologies. The truth is that, as our applications gain size and importance, we need to take security as seriously as automakers are required to.
Do you want to be responsible for the next theft of millions of credit card numbers or multimillion dollar losing e-commerce site crash? If you don't want to have to deal with the guilt and embarrassment which no amount of finger pointing will assuage, then you must make security a part of your development process today.
Wednesday, October 1, 2008
Does CAPTCHA Have a Future?
ZDNet and other tech sites are reporting that Microsoft's Hotmail CAPTCHA (Completely Automated Public Turing test to Tell Computers and Humans Apart) has been under attack again, even after the software giant made some changes to reduce the chances of spammers breaking through. The report says the CAPTCHA busting techniques used by the attackers have had a success rate as high as 15% that's spammer heaven!
Incidents such as these have caused a lot of people to write off CAPTCHA. Unfortunately, unless a company wants to have humans pouring through questionable submissions, there really is no other alternative. Audio CAPTCHA (which is less an alternative than a complement), I am afraid, is actually easier to break than printed CAPTCHA.
I submit that the issue is creativity. Spammers and hackers have busted CAPTCHA more by grunt than by smarts. The odds are somewhat in their favor since they need not have a 100% success rate, while their victims have to be able to beat them every time.
Just like with authentication and credentials, CAPTCHA by itself is breakable with relative ease. However, this ease can be reduced by several orders of magnitude if we increase the number of factors that would generate a positive remember two-factor authentication?
I have read that many spammers have actually used HAC (Human Assisted Computing) to break CAPTCHA. All the spammers have to do is setup a front site (offering things like movie downloads or pornography) and use the target CAPTCHA as if it were their own own. When the user of the front site passes the CAPTCHA test, the spammer immediately gains access to the target site.
This can be fought in many ways. Right off the bat, you probably want to limit or block image linking from external sites. If this is not possible, you could submit the URL of the form along with the user's CAPTCHA input. Any external site shenanigan will be detected with a simple URL match check.
Beyond getting more from CAPTCHA as it exists today, I would suggest improving on the current technology. For example, a photograph accompanied by questions only a human could answer, such as "which person in this picture appears to be youngest?" Another idea would be to use a short movie and ask the person to describe what is going on, or what action preceded what other action.
Another idea would be to analyze the users behavior to determine if it is a human or a machine. Following mouse movements for as little as 1 second can tell whether a hand is human or a simulated. Most scripts do not produce any mouse movement and if they do it is likely to be stilted.
Another thing technologists and developers should remember is that CAPTCHA is more a concept than a technology like artificial intelligence. As long as the objective is achieved we can call whatever we do CAPTCHA.
As computers become more sophisticated, we will have to come up with new Shibboleths to entrap non-human users. All it will take is us staying in touch with our creative sides.
Incidents such as these have caused a lot of people to write off CAPTCHA. Unfortunately, unless a company wants to have humans pouring through questionable submissions, there really is no other alternative. Audio CAPTCHA (which is less an alternative than a complement), I am afraid, is actually easier to break than printed CAPTCHA.
I submit that the issue is creativity. Spammers and hackers have busted CAPTCHA more by grunt than by smarts. The odds are somewhat in their favor since they need not have a 100% success rate, while their victims have to be able to beat them every time.
Just like with authentication and credentials, CAPTCHA by itself is breakable with relative ease. However, this ease can be reduced by several orders of magnitude if we increase the number of factors that would generate a positive remember two-factor authentication?
I have read that many spammers have actually used HAC (Human Assisted Computing) to break CAPTCHA. All the spammers have to do is setup a front site (offering things like movie downloads or pornography) and use the target CAPTCHA as if it were their own own. When the user of the front site passes the CAPTCHA test, the spammer immediately gains access to the target site.
This can be fought in many ways. Right off the bat, you probably want to limit or block image linking from external sites. If this is not possible, you could submit the URL of the form along with the user's CAPTCHA input. Any external site shenanigan will be detected with a simple URL match check.
Beyond getting more from CAPTCHA as it exists today, I would suggest improving on the current technology. For example, a photograph accompanied by questions only a human could answer, such as "which person in this picture appears to be youngest?" Another idea would be to use a short movie and ask the person to describe what is going on, or what action preceded what other action.
Another idea would be to analyze the users behavior to determine if it is a human or a machine. Following mouse movements for as little as 1 second can tell whether a hand is human or a simulated. Most scripts do not produce any mouse movement and if they do it is likely to be stilted.
Another thing technologists and developers should remember is that CAPTCHA is more a concept than a technology like artificial intelligence. As long as the objective is achieved we can call whatever we do CAPTCHA.
As computers become more sophisticated, we will have to come up with new Shibboleths to entrap non-human users. All it will take is us staying in touch with our creative sides.
Subscribe to:
Posts (Atom)