If your development shop is to improve in technology and quality while outpacing the competition, you need some amount of research and development (R&D). This is especially crucial in shops seeking to expand their intellectual property portfolio.
However, the word on the street is that R&D is only for very large corporations such as IBM or Microsoft with billions to spare. I used to feel that way too, but I was pleasantly surprised to learn from the best in the business that there was a cheaper alternative.
It turns out that Google, as in so many other areas, has blazed that trail. Not by inventing the concept, but by proving that it could be done. The idea of giving developers a percentage of their bench time to pursue whatever idea they fancy had been floating around for a good long while but had been flatly rejected by others.
I recall a certain project manager who would say "we all want to have fun, but we got bills to pay." This appears to be the rationale in most development shops, especially if there are budgetary constraints. However, I would like to propose a way to get around this hurdle fairly painlessly.
The first step is to improve in the time management department. It is surprising how poorly time is managed many development shops. For example, although there are long days at crunch time, between patches and releases, time killing is pointlessly common or filled with meetings of dubious utility. Methodologies such as Agile have resolved this imbalance somewhat, but we need to go even further.
Developer bench time can be optimized painlessly if done gradually. I would suggest beginning with a 5% allocation for personal projects. This is the equivalent to two one-hour meetings in a 40-hour week, barely felt but significant enough to establish the practice and explain the principle.
The key is to make sure this is not seen by developers as a break or social time. In fact, I would suggest two rules to make sure what needs to happen, happens during this special time:
(1) No web surfing except for brief visits to coding sites
(2) Heads down coding throughout — no pressure on the quality or the amount though.
Do not expect much to happen the first couple of weeks except for some habituation. However, if this schedule is kept religiously and developers are asked to speak briefly about their "personal projects" during the warm-up for regularly scheduled meetings, expect to start seeing some incredible code being generated by week number three.
The beauty of this practice, is that you will start seeing improvement in the quality of the production code and even an increase in collaboration among the developers.
As you start reaping the benefits of this program and your team's productivity grows —by virtue of their becoming better developers and by virtue of some of their personal project code finding its way into released software— it will be easier to sell to management the idea of expanding personal project time to a healthy 20%.
But, be patient enough to do so gradually. It is okay if it takes two years to go from 5% to 20%. The practice will yield fruits as long as it is done properly.
So there you have it: a very simple way in which your shop can benefit from R&D without breaking the bank. You are welcome.
Wednesday, March 25, 2009
Tuesday, March 24, 2009
The Learning Organization And Why You Need One
Many people in management believe a learning organization is one of those things that are "nice to have" and will be gotten around to if there is time or money available.
Worse yet, many think the phrase represents just another useless fad and want nothing to do with such a concept.
I will assume the worse and try to sell you on the idea before telling you how to get there.
If the sum total of your company's expertise is no more than than that of a single employee —even if that employee is the genius founder— then your organization doesn't have much of a future. The world becomes increasingly complex as time advances and your firm needs to keep up.
So, what are good examples of learning organizations? Google, HP, Apple, Walmart, Microsoft are just a few of the companies that in one way or another may be characterized as learning organizations. Their balance sheets and market positioning attest to the value of the knowledge based shop. But enough of the preliminaries. Lets begin by defining a learning organization.
The beginning and end of a learning organization is the realization that knowledge is valuable asset that is as worthy of safeguarding as any other capital good. In fact, it is fair to say that knowledge is probably the most important asset, and I am not just talking about intellectual property or patents, which is more of a legal construct.
Having established that the valuation of organizational knowledge is the mark of a learning organization and the secret of its success, let us examine the ways you can make your organization a learning one. After much thought I realized that five action words pretty much sum up the attitudes of knowledge shop:
1) Document
2) Train
3) Audit
4) Preserve
5) Transition
Document
I have to be careful when I say "document" because many companies bury their employees in red tape and procedural straight jackets in the name of "documentation."
No, no, no! In fact, hell, no!
Documentation does not mean filling out forms in triplicate or writing thick hundred page reports with useless details that nobody will ever read. This is is actually a productivity killer.
The documentation that works is the kind that fits easily into the workflow and is not onerous. In fact the best way to get your developers documenting their work is to give them a say in which is the best way to document their work.
A easy exercise is to have developers perform a specific task which they document the best way they can think of. Then have them pass on their notes to another developer who is supposed to replicate the task using only the written instructions supplied by the first developer. Then ask the second developer to write suggestions to the first developer as to how he/she could improve the original set of instructions he or she got.
The best thing about this exercise is that after discussing the results you can put together a documentation guide that everyone will be motivated to use because they are sold on its usefulness. After that, it will be a trivial matter to decide what software to use if none has been chosen yet.
Train
In a true learning organization, all members should either train or be trained at least once every week. This should be written into the contract or the SOP.
Of course, this requirement should be flexible in every aspect. All forms of interactive knowledge transfer should be accommodated : group sessions, learning lunch pair ups, one-on-one cubicle tutorials, conference calls, any which way as long as it allows for note taking and questions.
Although blogs, wikis, instant messages and even tweets are good, they should not be allowed to take the place of actual interactive training.
Also, some amount of learning verification or testing should be worked into the process. This need not be too formal or extensive, but there needs to be a way to gauge what forms of training work best and which don't.
Audit
This is probably one of the most important and yet complex aspects of keeping an organization ahead in the age of information. In the NFL (National Football League) they call it "studying the game film."
Every release, major patch, platform change or post-deployment emergency should be audited internally. It is not only important to have a post-mortem when things go bad; there should be one when they go well too.
Retrospective analysis needs to become part of the workflow. This not only insures improved performance and better quality products; it also helps to solidify winning practices and avoid repeat failures. Think of this as insurance.
Retrospectives will not hold back the team if they are properly planned. Also, if one of the goals of the audit is to make the next release less painful, and the promise is delivered on, the team as a whole will gladly rally behind such reviews.
Preserve
If you have a winning horse, you insure it and feed it the best purina has to offer. You need to preserve proven practices and technologies that work for your team. If these are proprietary and have obvious commercial value, your legal department will probably make sure you do.
However, anything that can help your team and others in the future or around the organization should be preserved and/or disseminated. One way to do this is through a developer intranet knowledge base or wiki.
Care must be taken, however, to provide a proper way of cataloguing or searching this information. Also, it might be a good idea to separate or differentiate preliminary notes from actual finalized data.
Another very important way to preserve knowledge is through enterprise code libraries and APIs. However, no library or API should be allowed be catalogued without proper documentation and source code — this is extremely important especially in these litigious times.
Transition
A learning organization is dynamic and forward looking. This means that it is important to be always planning for the next stage or technology. Although I don't suggest that every new technological fad be followed, I don't think it is healthy for a shop to keep old code around for so long that it becomes too difficult or impractical to support — remember all that old code that needed updating when the year 2000 rolled around?
Be cutting edge, but not bleeding edge. Whenever possible, use the latest stable version of any technology you choose and plan for transition far in advance. Most importantly, go at a pace that suits your organizational needs not, your vendor's sales schedule.
Speaking of transition, technology moves are easier when design objectives are formulated in terms of functionality needed as opposed to technology to be used.
For example, instead of having as a requirement that says "the SAX API should be used for this task ", your stated objective should read something like "a serial access XML parser that is efficient with large data streams or documents should be used." If you do mention a specific implementation, do so parenthetically (as an example).
A learning organization stays ahead because it looks ahead without forgetting the lessons learned along the way. It harnesses (documents) and multiplies (trains) its experiential knowledge. It gets better with time because it maintains a proper feedback loop (audits). It performs optimally because it knows its strengths. It keeps around and sharpens (preserves) its best tools in order to reuse them for even bigger kills. And, lastly, it knows when to change horses (transfers).
Worse yet, many think the phrase represents just another useless fad and want nothing to do with such a concept.
I will assume the worse and try to sell you on the idea before telling you how to get there.
If the sum total of your company's expertise is no more than than that of a single employee —even if that employee is the genius founder— then your organization doesn't have much of a future. The world becomes increasingly complex as time advances and your firm needs to keep up.
So, what are good examples of learning organizations? Google, HP, Apple, Walmart, Microsoft are just a few of the companies that in one way or another may be characterized as learning organizations. Their balance sheets and market positioning attest to the value of the knowledge based shop. But enough of the preliminaries. Lets begin by defining a learning organization.
The beginning and end of a learning organization is the realization that knowledge is valuable asset that is as worthy of safeguarding as any other capital good. In fact, it is fair to say that knowledge is probably the most important asset, and I am not just talking about intellectual property or patents, which is more of a legal construct.
Having established that the valuation of organizational knowledge is the mark of a learning organization and the secret of its success, let us examine the ways you can make your organization a learning one. After much thought I realized that five action words pretty much sum up the attitudes of knowledge shop:
1) Document
2) Train
3) Audit
4) Preserve
5) Transition
Document
I have to be careful when I say "document" because many companies bury their employees in red tape and procedural straight jackets in the name of "documentation."
No, no, no! In fact, hell, no!
Documentation does not mean filling out forms in triplicate or writing thick hundred page reports with useless details that nobody will ever read. This is is actually a productivity killer.
The documentation that works is the kind that fits easily into the workflow and is not onerous. In fact the best way to get your developers documenting their work is to give them a say in which is the best way to document their work.
A easy exercise is to have developers perform a specific task which they document the best way they can think of. Then have them pass on their notes to another developer who is supposed to replicate the task using only the written instructions supplied by the first developer. Then ask the second developer to write suggestions to the first developer as to how he/she could improve the original set of instructions he or she got.
The best thing about this exercise is that after discussing the results you can put together a documentation guide that everyone will be motivated to use because they are sold on its usefulness. After that, it will be a trivial matter to decide what software to use if none has been chosen yet.
Train
In a true learning organization, all members should either train or be trained at least once every week. This should be written into the contract or the SOP.
Of course, this requirement should be flexible in every aspect. All forms of interactive knowledge transfer should be accommodated : group sessions, learning lunch pair ups, one-on-one cubicle tutorials, conference calls, any which way as long as it allows for note taking and questions.
Although blogs, wikis, instant messages and even tweets are good, they should not be allowed to take the place of actual interactive training.
Also, some amount of learning verification or testing should be worked into the process. This need not be too formal or extensive, but there needs to be a way to gauge what forms of training work best and which don't.
Audit
This is probably one of the most important and yet complex aspects of keeping an organization ahead in the age of information. In the NFL (National Football League) they call it "studying the game film."
Every release, major patch, platform change or post-deployment emergency should be audited internally. It is not only important to have a post-mortem when things go bad; there should be one when they go well too.
Retrospective analysis needs to become part of the workflow. This not only insures improved performance and better quality products; it also helps to solidify winning practices and avoid repeat failures. Think of this as insurance.
Retrospectives will not hold back the team if they are properly planned. Also, if one of the goals of the audit is to make the next release less painful, and the promise is delivered on, the team as a whole will gladly rally behind such reviews.
Preserve
If you have a winning horse, you insure it and feed it the best purina has to offer. You need to preserve proven practices and technologies that work for your team. If these are proprietary and have obvious commercial value, your legal department will probably make sure you do.
However, anything that can help your team and others in the future or around the organization should be preserved and/or disseminated. One way to do this is through a developer intranet knowledge base or wiki.
Care must be taken, however, to provide a proper way of cataloguing or searching this information. Also, it might be a good idea to separate or differentiate preliminary notes from actual finalized data.
Another very important way to preserve knowledge is through enterprise code libraries and APIs. However, no library or API should be allowed be catalogued without proper documentation and source code — this is extremely important especially in these litigious times.
Transition
A learning organization is dynamic and forward looking. This means that it is important to be always planning for the next stage or technology. Although I don't suggest that every new technological fad be followed, I don't think it is healthy for a shop to keep old code around for so long that it becomes too difficult or impractical to support — remember all that old code that needed updating when the year 2000 rolled around?
Be cutting edge, but not bleeding edge. Whenever possible, use the latest stable version of any technology you choose and plan for transition far in advance. Most importantly, go at a pace that suits your organizational needs not, your vendor's sales schedule.
Speaking of transition, technology moves are easier when design objectives are formulated in terms of functionality needed as opposed to technology to be used.
For example, instead of having as a requirement that says "the SAX API should be used for this task ", your stated objective should read something like "a serial access XML parser that is efficient with large data streams or documents should be used." If you do mention a specific implementation, do so parenthetically (as an example).
A learning organization stays ahead because it looks ahead without forgetting the lessons learned along the way. It harnesses (documents) and multiplies (trains) its experiential knowledge. It gets better with time because it maintains a proper feedback loop (audits). It performs optimally because it knows its strengths. It keeps around and sharpens (preserves) its best tools in order to reuse them for even bigger kills. And, lastly, it knows when to change horses (transfers).
Sunday, March 22, 2009
IT Spending In Hard Times
Many firms are seeing hard times as certain sectors of the economy have been shrinking over the past few months. This is, of course, not a desirable condition. However, I would like to posit that a lot of good could come from the belt-tightening measures many an IT department will have to go through.
Actually, I should probably begin by stating that there is good belt-tightening and there is bad. Companies that engage in the latter my find themselves in an even worse crisis or, worse yet, may even go out of business.
As IT people, we usually have less control over how much of the company's budget is allocated to our department than how we will allocate the funds we do end up with.
Let's examine different key items on an IT budget and see how these can be kept up to par, if not improved during lean times.
Hardware
It is often assumed that if hardware is not the absolute latest (1) performance will suffer and (2) employee morale will go down. While we should not fall so far behind as to isolate the company technologically, hardware purchases should be justifiable in relation to the job requirement of the employee in question.
For example, studies have show that developers and designers improve their performance when using dual monitors such allocation for these types of workers is justified. However, executives and administrative assistants have no use for such extra spending so IT can save by simply letting them have a single, reasonably sized monitor.
By the same token, there might be other items, such as Blackberries and corporate cell phone accounts, that many developers really have no need for and just represent extra spending — exceptions must be made of course for product engineers who might need to be on call.
Software
This is indeed a great time to look into free tools and open source software. I am not advocating necessarily a wholesale conversion, just that areas of potential savings be identified.
Another, very important area to address is duplication. Many companies, for example, will spend money on site licenses for software they already have in another form. A good example which I have noticed at companies who've used my services is compression software. I have found it ridiculous that a company would spend money to deploy an operating system that has built in compression support and still spend money on a third party compression utility.
Projects
Most companies are good at killing projects that are deemed unprofitable or useless. However, many companies support parallel efforts at great expense.
Sometimes, it is just a matter of ignorance; at other times, it has to do with inter-departmental rivalry. Either way, it is wasteful for there to be duplicated development efforts within an enterprise. Any effort to curtail this kind of thing is worth the expense.
I would suggest a corporate version of SourceForge where all enterprise applications are catalogued and managers can search by keyword or description for applications that they need.
Division of Labor
There is a lot that can be said about this. I will try to be brief. Many companies, respond to crises by either laying-off developers and admins or by getting rid of contractors. While, there are good arguments to be made for either move in individual cases, as fixed policies both are horrendous ideas.
Let's first examine the basics. No matter how good times ever were, the number of full-time developers or network administrators should have been kept to a minimum. There are two reasons for this: (1) we do not want people who have so little to do they get bored and (2) software engineers are the kind of employee you want to have for a very long time as their value to the company increases exponentially as their seniority accrues.
Contractors should be hired to fill temporary gaps in personnel or to supplement the core team in large projects. The decision to hire contractors should always be based on potential saving and should be in no way connected to whether there is a lot of IT dollars to waste.
In other words, if you had to either cut contracts or lay off developers due to the recent economic downturn, you were probably doing something wrong all along,
Amenities
These tend to vary a lot and it's hard to tell a firm that they should remain constant. However, I believe that, if the amenities in question were properly analyzed or justified initially, they would be less likely to be subject to natural variations in the IT budget.
Another thing to be careful of is to mistake necessities for amenities. Office supplies, lights, proper seating and desk space are not amenities they are job requirements — ask OSHA.
Coffee machines, food concessions, free snacks, cable TV in the break room, gym memberships, etc, are actual amenities.
There is no easy way to dispense with freebies employees have grown accustomed to. This is why I always recommend that companies offer just a few choice amenities which they are likely to be able to maintain regardless of the vagaries of the economy.
Another tack would to have employees vote on what amenities they wish to keep and which they are willing to part with. This would help to keep morale high.
Efficiency
These tough economic times are the best encouragement any IT department can have to streamline its operations. If something can be achieved in less steps, reduce the number of steps.
Minimize requirements, simplify procedures, consolidate functions, look into virtualization if you haven't before, do not underestimate hardware recycling or reuse, try not to reinvent the wheel every time, cut costs wisely and where it is least felt or less likely to be seen.
This current crisis is not the first one we have faced and is not likely to be the last. It can be an opportunity to shape our organizations for a magnificent resurgence.
Actually, I should probably begin by stating that there is good belt-tightening and there is bad. Companies that engage in the latter my find themselves in an even worse crisis or, worse yet, may even go out of business.
As IT people, we usually have less control over how much of the company's budget is allocated to our department than how we will allocate the funds we do end up with.
Let's examine different key items on an IT budget and see how these can be kept up to par, if not improved during lean times.
Hardware
It is often assumed that if hardware is not the absolute latest (1) performance will suffer and (2) employee morale will go down. While we should not fall so far behind as to isolate the company technologically, hardware purchases should be justifiable in relation to the job requirement of the employee in question.
For example, studies have show that developers and designers improve their performance when using dual monitors such allocation for these types of workers is justified. However, executives and administrative assistants have no use for such extra spending so IT can save by simply letting them have a single, reasonably sized monitor.
By the same token, there might be other items, such as Blackberries and corporate cell phone accounts, that many developers really have no need for and just represent extra spending — exceptions must be made of course for product engineers who might need to be on call.
Software
This is indeed a great time to look into free tools and open source software. I am not advocating necessarily a wholesale conversion, just that areas of potential savings be identified.
Another, very important area to address is duplication. Many companies, for example, will spend money on site licenses for software they already have in another form. A good example which I have noticed at companies who've used my services is compression software. I have found it ridiculous that a company would spend money to deploy an operating system that has built in compression support and still spend money on a third party compression utility.
Projects
Most companies are good at killing projects that are deemed unprofitable or useless. However, many companies support parallel efforts at great expense.
Sometimes, it is just a matter of ignorance; at other times, it has to do with inter-departmental rivalry. Either way, it is wasteful for there to be duplicated development efforts within an enterprise. Any effort to curtail this kind of thing is worth the expense.
I would suggest a corporate version of SourceForge where all enterprise applications are catalogued and managers can search by keyword or description for applications that they need.
Division of Labor
There is a lot that can be said about this. I will try to be brief. Many companies, respond to crises by either laying-off developers and admins or by getting rid of contractors. While, there are good arguments to be made for either move in individual cases, as fixed policies both are horrendous ideas.
Let's first examine the basics. No matter how good times ever were, the number of full-time developers or network administrators should have been kept to a minimum. There are two reasons for this: (1) we do not want people who have so little to do they get bored and (2) software engineers are the kind of employee you want to have for a very long time as their value to the company increases exponentially as their seniority accrues.
Contractors should be hired to fill temporary gaps in personnel or to supplement the core team in large projects. The decision to hire contractors should always be based on potential saving and should be in no way connected to whether there is a lot of IT dollars to waste.
In other words, if you had to either cut contracts or lay off developers due to the recent economic downturn, you were probably doing something wrong all along,
Amenities
These tend to vary a lot and it's hard to tell a firm that they should remain constant. However, I believe that, if the amenities in question were properly analyzed or justified initially, they would be less likely to be subject to natural variations in the IT budget.
Another thing to be careful of is to mistake necessities for amenities. Office supplies, lights, proper seating and desk space are not amenities they are job requirements — ask OSHA.
Coffee machines, food concessions, free snacks, cable TV in the break room, gym memberships, etc, are actual amenities.
There is no easy way to dispense with freebies employees have grown accustomed to. This is why I always recommend that companies offer just a few choice amenities which they are likely to be able to maintain regardless of the vagaries of the economy.
Another tack would to have employees vote on what amenities they wish to keep and which they are willing to part with. This would help to keep morale high.
Efficiency
These tough economic times are the best encouragement any IT department can have to streamline its operations. If something can be achieved in less steps, reduce the number of steps.
Minimize requirements, simplify procedures, consolidate functions, look into virtualization if you haven't before, do not underestimate hardware recycling or reuse, try not to reinvent the wheel every time, cut costs wisely and where it is least felt or less likely to be seen.
This current crisis is not the first one we have faced and is not likely to be the last. It can be an opportunity to shape our organizations for a magnificent resurgence.
Tuesday, March 17, 2009
Why We Fight
This phrase has become very popular over the past few years in connection with the Iraq war. I however would like to turn this question on developers : why do we code? What is our job?
Before you dismiss the question as simplistic, or irrelevant, or both, think about it. What is your deliverable at the end of your project?
If you want to build a ship, don't drum up the men to gather wood, divide the work and give orders. Instead, teach them to yearn for the vast and endless sea.
—Antoine de Saint-Exupery
Lack of clarity in this respect has ruined many a software engineering job. Of course, I am not trying to say that any outfit can afford to dispense with specialization or division of labor. In a large project no one person can execute every aspect of the job at ever stage of the development cycle.
But, does this mean that we can afford to disengage? Not at all. On the contrary, we should be all the more plugged into the process and and should have the goal clear in our minds.
To truly contribute to a project, developers need to be flexible and open-minded. This might mean enduring temporary discomfort to achieve the greater joy of a solid product—: and, no, I am not talking about pulling all nighters although I do not condemn the practice per se.
The engagement I recommend is a commitment to the final product which would transcend pet peeves and preferences, a willingness to learn, adapt, go around obstacles and eschew pettiness. Given the large egos we IT people have, I know this is not easy; but we will have to if we are to remain relevant.
Before you dismiss the question as simplistic, or irrelevant, or both, think about it. What is your deliverable at the end of your project?
If you want to build a ship, don't drum up the men to gather wood, divide the work and give orders. Instead, teach them to yearn for the vast and endless sea.
—Antoine de Saint-Exupery
Lack of clarity in this respect has ruined many a software engineering job. Of course, I am not trying to say that any outfit can afford to dispense with specialization or division of labor. In a large project no one person can execute every aspect of the job at ever stage of the development cycle.
But, does this mean that we can afford to disengage? Not at all. On the contrary, we should be all the more plugged into the process and and should have the goal clear in our minds.
To truly contribute to a project, developers need to be flexible and open-minded. This might mean enduring temporary discomfort to achieve the greater joy of a solid product—: and, no, I am not talking about pulling all nighters although I do not condemn the practice per se.
The engagement I recommend is a commitment to the final product which would transcend pet peeves and preferences, a willingness to learn, adapt, go around obstacles and eschew pettiness. Given the large egos we IT people have, I know this is not easy; but we will have to if we are to remain relevant.
Sunday, March 15, 2009
The Beach On Which Many Come To Die
Reading through Slashdot messages recently I was reminded of the many vulnerabilities web-facing applications are prone to and how little thought is given to their security. After waging fierce battle on the server side, many applications make it to the shores of the Internet only to be shot down by hackers on speedboats.
This happens because poor practices coupled with many misconceptions translate into an incredibly porous security wall safeguarding applications.
The dangers of the web are many and are constantly evolving, so the secret in securing applications is to stick to principles rather than techniques or tricks that become obsolete in short order.
Principle 1 : Don't Give Away The Store
I am not advocating security through obscurity here — that would be very foolish. But I really think that not suppressing debugging error messages at least makes your site more tempting to a hacker — and may even make your application more vulnerable by airing too many details of your server configuration.
Principle 2 : Don't Trust Any Client To Validate or Scrub Data For You
I am a strong believer in client side validation, but I am extremely aware that clients are easily manipulated. Got that? EASILY MANIPULATED! This goes not just for form data, it also applies to cookies, AJAX requests and hidden fields [including the viewstate, for you .NET types].
Some years ago, I recall reading an interview with a famous hacker (I think it was Kevin Mitnick) who said that his favorite hacking tool was a web browser. That was several years ago. Browsers have become more sophisticated and powerful since.
In the trenches, I have had the opportunity to see first hand how much damage a skillful hacker can do using a browser. All this is to say that, the only reason to validate on the client side should be to offload server cycles and for a better user experience. There must be validation on the server side and it needs to be more rigorous.
Principle 3 : Remain Vigilant
Check your logs and raw data from time to time. Look for anomalous inputs or malware signatures. The hacker that failed to break in today is likely to try again tomorrow.
Subscribe to security bulletins. Learn about the vulnerabilities and attacks and keep an eye out for trouble. Nothing beats an early warning system.
Principle 4 : Think Ahead
Look for vulnerabilities in your own applications. It is a million times better for you to find a exploit than for a hacker to do so. Poke your code, be creative with QA scripts. Try to get white hat hackers to take a crack at it. This will pay off in the short and long run.
Principle 5 : Stay Up To Date
In as much as practicable, keep your libraries and compilers up to date. Don't overdo it so much as to be running Beta builds either. I like to say be "cutting edge" not "bleeding edge."
Use the latest stable versions of your platforms of choice, but be careful not to be faddish. Faddish shops also get hacked because they venture into using untested products whose vulnerabilities may not even be known to the manufacturers yet.
In Sum
These five principles, applied to every stage of software development and subsequent maintenance will allow you to guarantee your users a secure application with outstanding uptime.
This happens because poor practices coupled with many misconceptions translate into an incredibly porous security wall safeguarding applications.
The dangers of the web are many and are constantly evolving, so the secret in securing applications is to stick to principles rather than techniques or tricks that become obsolete in short order.
Principle 1 : Don't Give Away The Store
I am not advocating security through obscurity here — that would be very foolish. But I really think that not suppressing debugging error messages at least makes your site more tempting to a hacker — and may even make your application more vulnerable by airing too many details of your server configuration.
Principle 2 : Don't Trust Any Client To Validate or Scrub Data For You
I am a strong believer in client side validation, but I am extremely aware that clients are easily manipulated. Got that? EASILY MANIPULATED! This goes not just for form data, it also applies to cookies, AJAX requests and hidden fields [including the viewstate, for you .NET types].
Some years ago, I recall reading an interview with a famous hacker (I think it was Kevin Mitnick) who said that his favorite hacking tool was a web browser. That was several years ago. Browsers have become more sophisticated and powerful since.
In the trenches, I have had the opportunity to see first hand how much damage a skillful hacker can do using a browser. All this is to say that, the only reason to validate on the client side should be to offload server cycles and for a better user experience. There must be validation on the server side and it needs to be more rigorous.
Principle 3 : Remain Vigilant
Check your logs and raw data from time to time. Look for anomalous inputs or malware signatures. The hacker that failed to break in today is likely to try again tomorrow.
Subscribe to security bulletins. Learn about the vulnerabilities and attacks and keep an eye out for trouble. Nothing beats an early warning system.
Principle 4 : Think Ahead
Look for vulnerabilities in your own applications. It is a million times better for you to find a exploit than for a hacker to do so. Poke your code, be creative with QA scripts. Try to get white hat hackers to take a crack at it. This will pay off in the short and long run.
Principle 5 : Stay Up To Date
In as much as practicable, keep your libraries and compilers up to date. Don't overdo it so much as to be running Beta builds either. I like to say be "cutting edge" not "bleeding edge."
Use the latest stable versions of your platforms of choice, but be careful not to be faddish. Faddish shops also get hacked because they venture into using untested products whose vulnerabilities may not even be known to the manufacturers yet.
In Sum
These five principles, applied to every stage of software development and subsequent maintenance will allow you to guarantee your users a secure application with outstanding uptime.
IT Myths
The IT, world like many other environs, is full of urban legends. We sometimes call them FUD and sometimes we call them "commissioned studies". Some are older than others, but every now and then they come around to haunt us in one form or another.
Linux is Free
This myth is spread by many well-intentioned free software advocates unaware that such blatant lie only harms its chances of adoption. Like with most myths there is some truth to this one: Linux is free for individuals who wish to experiment with it, but not for enterprises.
Even if a company decides to download free binaries from the Internet, switching to Linux will have its costs. The question to ask is how much less will it cost than choosing the alternative. Linux is a fix-and-forget kind of OS solution whose cost of adoption is biggest upfront — but with reduced maintenance over the life of its deployment. This fact sometimes leads many to conclude that it's adoption is costlier when it might in fact be significantly lower than a competing deployment whose upfront costs are nominally lower.
Windows OSs Have The Highest ROI and/or Lowest TCO
This one has been expressed by a number of "consulting" firms in their "commission studies" and is the result of creative number crunching. I don't mean necessarily that Windows OS costs are the worst in the business, just that they are not the shoo-ins many suggest.
For one, it is patently dishonest to suggest that all Microsoft to Microsoft transitions are friction free and require little or no re-training — wake up and smell the Vista, man! This is also an issue with Office 2003 and Office 2007 — fortunately there is third-party software to help with that one.
Another bit of dishonesty occurs when "consultants" fail to work into the Windows OS TCO/ROI the labor costs of things like Patch Tuesday as well as the high cost of antivirus software.
Macs Are Expensive
Again, the only way one can make such an assertion is to only factor in the sticker price. However such an assertion is the equivalent of suggesting to the police department that they could save on costs by dropping the training requirement from their recruits.
Macs may have a higher sticker price, but have lower maintenance costs than PCs and have a longer useful life. This needs to be factored into any serious study.
Now That We Have Java/C#, Who Needs C/C++?
Back in the 80s there were those using this same kind of logic to predict the demise of the different flavors of Assembler. Now the the tunnel visionaries have set their sites on virtual machine code. Please, guys, stop embarrassing yourselves. Another similar claim is the prediction made almost yearly that UML will replace all computer languages — please!
Relational Databases Are On Their Way Out
I think I've been hearing this one for the past 10 years. The telling part is that those who usually say such things have never had to setup a data store. Relational databases not only reflect the way we think about things, but they also allow us some of the best ways to scale massive amounts of information.
Open Source Software Is Not Fit For The Enterprise
This is one is usually spread by some well-know very ridiculous actors. The interesting part is that one of the most important professions in the world, Civil Engineering, follows pretty much an open source model. Anyone wants to say that Civil Engineering is not fit for the enterprise?
The Bottom Line
Information Technology is supposed to be based on science. Heeding rumors leads many companies to burn large amounts of money on useless pursuits. In the business world the most successful companies make solid IT investments and the losers fumble in this area. Success means not only thinking outside the box, but listening outside the echo chamber as well.
Linux is Free
This myth is spread by many well-intentioned free software advocates unaware that such blatant lie only harms its chances of adoption. Like with most myths there is some truth to this one: Linux is free for individuals who wish to experiment with it, but not for enterprises.
Even if a company decides to download free binaries from the Internet, switching to Linux will have its costs. The question to ask is how much less will it cost than choosing the alternative. Linux is a fix-and-forget kind of OS solution whose cost of adoption is biggest upfront — but with reduced maintenance over the life of its deployment. This fact sometimes leads many to conclude that it's adoption is costlier when it might in fact be significantly lower than a competing deployment whose upfront costs are nominally lower.
Windows OSs Have The Highest ROI and/or Lowest TCO
This one has been expressed by a number of "consulting" firms in their "commission studies" and is the result of creative number crunching. I don't mean necessarily that Windows OS costs are the worst in the business, just that they are not the shoo-ins many suggest.
For one, it is patently dishonest to suggest that all Microsoft to Microsoft transitions are friction free and require little or no re-training — wake up and smell the Vista, man! This is also an issue with Office 2003 and Office 2007 — fortunately there is third-party software to help with that one.
Another bit of dishonesty occurs when "consultants" fail to work into the Windows OS TCO/ROI the labor costs of things like Patch Tuesday as well as the high cost of antivirus software.
Macs Are Expensive
Again, the only way one can make such an assertion is to only factor in the sticker price. However such an assertion is the equivalent of suggesting to the police department that they could save on costs by dropping the training requirement from their recruits.
Macs may have a higher sticker price, but have lower maintenance costs than PCs and have a longer useful life. This needs to be factored into any serious study.
Now That We Have Java/C#, Who Needs C/C++?
Back in the 80s there were those using this same kind of logic to predict the demise of the different flavors of Assembler. Now the the tunnel visionaries have set their sites on virtual machine code. Please, guys, stop embarrassing yourselves. Another similar claim is the prediction made almost yearly that UML will replace all computer languages — please!
Relational Databases Are On Their Way Out
I think I've been hearing this one for the past 10 years. The telling part is that those who usually say such things have never had to setup a data store. Relational databases not only reflect the way we think about things, but they also allow us some of the best ways to scale massive amounts of information.
Open Source Software Is Not Fit For The Enterprise
This is one is usually spread by some well-know very ridiculous actors. The interesting part is that one of the most important professions in the world, Civil Engineering, follows pretty much an open source model. Anyone wants to say that Civil Engineering is not fit for the enterprise?
The Bottom Line
Information Technology is supposed to be based on science. Heeding rumors leads many companies to burn large amounts of money on useless pursuits. In the business world the most successful companies make solid IT investments and the losers fumble in this area. Success means not only thinking outside the box, but listening outside the echo chamber as well.
Wednesday, March 4, 2009
Errata and More Musings On Open Source
In my last posting I pitted open source developers against "commercial software developers." Then, someone pointed out to me that many open source projects are commercial and their developers are no less competitive than the cube dwellers at Adobe or Microsoft. So, I apologize for using such imprecise language.
Back to coding. I don't think I emphasized enough in my last post how much eating of one's own dog food goes on in the open source world.
A shining example is the Apache Foundation. Their product is rock solid because they pound it probably more than their most demanding users — I guess their deal with IBM didn't hurt either, but they were good long before that.
Some proprietary software companies have tentatively gotten some of their developers to go into the trenches and observe their users in a natural setting and/or become users themselves.
No only do I think this is a nice "year on the farm" experience with incalculable returns in social currency, but I think it really makes developers better at what they do. Could it be that civil engineers are so good because they have to drive on their work to get to their offices?
Imagine how much insight you would gain on the quality of your POS software if you were to spend a week operating the checkout machine a local retail outlet. Such an experience is so richly textured and layered it almost defies description, except to say that it benefits all the parties involved.
Sadly, I have seen so many times cases where developers are not only not required to use the products they play a part in developing but they are almost forbidden to do so.
I am afraid we many times hurt our business through too much specialization. Not-my-department-ism is making us myopic and leading us down a path where barely have a theoretical grasp of what our software does.
In my work I have run into applications with very obvious bugs that only emerge during real-world use but are elusive to QA scripts.
Speaking of QA, we need to understand that Quality Assurance is an inexact science. As software becomes more complex and usage scenarios become more varied, the predictive nature of QA scripts becomes all the more lacking in reliability.
Real world testing, eating our own dog food, living off the land, whatever we call it is a necessary part of quality work. We need to use our products as users if we are to develop truly best-of-breed software.
Back to coding. I don't think I emphasized enough in my last post how much eating of one's own dog food goes on in the open source world.
A shining example is the Apache Foundation. Their product is rock solid because they pound it probably more than their most demanding users — I guess their deal with IBM didn't hurt either, but they were good long before that.
Some proprietary software companies have tentatively gotten some of their developers to go into the trenches and observe their users in a natural setting and/or become users themselves.
No only do I think this is a nice "year on the farm" experience with incalculable returns in social currency, but I think it really makes developers better at what they do. Could it be that civil engineers are so good because they have to drive on their work to get to their offices?
Imagine how much insight you would gain on the quality of your POS software if you were to spend a week operating the checkout machine a local retail outlet. Such an experience is so richly textured and layered it almost defies description, except to say that it benefits all the parties involved.
Sadly, I have seen so many times cases where developers are not only not required to use the products they play a part in developing but they are almost forbidden to do so.
I am afraid we many times hurt our business through too much specialization. Not-my-department-ism is making us myopic and leading us down a path where barely have a theoretical grasp of what our software does.
In my work I have run into applications with very obvious bugs that only emerge during real-world use but are elusive to QA scripts.
Speaking of QA, we need to understand that Quality Assurance is an inexact science. As software becomes more complex and usage scenarios become more varied, the predictive nature of QA scripts becomes all the more lacking in reliability.
Real world testing, eating our own dog food, living off the land, whatever we call it is a necessary part of quality work. We need to use our products as users if we are to develop truly best-of-breed software.
Monday, March 2, 2009
Why You Should Watch Open Source Even If You Don't Care For It
Although most of the software I have written is what could be characterized as proprietary, I have followed many open source projects with interest and have come to admire not just the quality of the code but the process whereby it comes into existence.
Developers whose disdain for open source prevents them from coming into contact with it cheat themselves. This is hardly new knowledge; however, my deductions and learning from open source &0151; in as much as they reflect my unique circumstances and propinquities— may prove useful to you, so I invite you to read on.
Refactoring
People who write commercial software, even those who are the staunchest advocates of refactoring, do not give refactoring its dues. Even in the most refactoring-friendly shops, this is done as a one-off process and never acknowledged as benefitting more than the project at hand.
In the open source world however, refactoring is given more attention because, to paraphrase the famous saying, refactoring anywhere is boost to good code everywhere. My favorite example is how the Linux kernel code has been systematically rewritten even when, as a product, it has consistently outperformed its nearest competitor for a very long time.
So, why do open source developers refactor more? DaVinci-like artistic pride in their work (which is always open for the world to examine) is a prima facie motive.
However, I think there is also a realization among these developers that by improving on earlier work, they improve on their skills as developers and hence their chances of writing better code in the future.
This, I think is a philosophical stance that any development team would profit from. Granted that there are release cycles and deadlines to satisfy, developers can still plan for refactoring. This might be achieved by regularizing schedules and avoiding, in as much as possible, the extremes that make developers go from consecutive all-nighters around release date to WOW-ridden time-killing days.
I know overtime and comp time are sometimes inevitable, but they should be the exception to the rule. A good project manager or head developer should plan for professional development during bench time with as much care as he/she would budget project-earmarked time. Under such circumstances, enhanced refactoring can be worked in quite easily. Google's 20%-time rule is perhaps the most creative way to achieve this.
Code Review
In the commercial software world the very phrase code review has almost the same connotation as administrative leave except that the latter is less tedious. This is very unfortunate especially since it hurts quality and promotes more bugs than any other practice I can imagine.
Agile shops have helped to remove some of the stigma associated with the phrase, but it is the open source world that has led the way. Open source developers welcome code review the way painters prepare to display their work in a swanky gallery. Why is this?
After monitoring discussion threads for couple open source projects, I have to come to realize that the reason for this confidence might stem from the following factors I've observed :
QA and Testing
Sometimes, inevitably so, despite the best efforts to separate testers from developers, there is pressure on the QA team to certify software too quickly. Understandably, this pressure to ship or release by a certain date is due to the fact that delays cost money.
However, a better practice would be to prioritize QA reports and provide graded certifications so that a product can still ship on time with an assurance given to customers that there will be reasonable number of updates included with the original price of the software product. Besides, updates and downloadable patches are quite inexpensive to provide over the Internet.
One reason many open source products have far fewer bugs than their commercial counterparts is that, in the open source world, bugs are sought feverishly and expected as part of the process. In the commercial world, however, I am afraid bugs are seen as nuisances and departures from "the original plan."
This transparency might seem self-defeating on the surface, imagine how devastating it would be if a disgruntled ex-employee were to leak the truth to the press!
Before anyone mentions Windows Vista, let me hasten to say that I do not condone the release of incomplete products to paying customers. Buyers have a right to expect a certain finish and polish to commercial software products, but they might be more likely to accept certain limitations with the understanding that there are a few minor fixes pending — remember the "we owe" sheet you had to sign at closing when you bought your last home?
Division of Labor
Project managers for both commercial and open source software projects subscribe to the principle of labor division. However, open source projects seem to benefit more from its application. Why is this?
It appears that open source projects are a lot more flexible in how they are structured. In other words, people are willing to drop previously assigned action items and take on new ones as circumstances dictate. Egos seem to be less hypersensitive and players are more flexible.
The reason commercial shops are so inflexible has less to do with IT that with HR. It is a shameful display of incompetence on the part of IT managers that they have not realized that HR titles don't always fit an individual's true capabilities or the specific needs of a project. I would be more forgiving of IT managers if there weren't other fields in which the same issue is handled in a much better way.
In the movie industry, for example, the director may play parts in a scene which, in turn, might be directed by another person. The finished movie is more important than nominal titles and no one feels cheated by such practical reassignments. Likewise, in the software industry, we need to be able to reorganize a team as many times and in as many ways as we need to create a truly best-of-breed product — titles be damned!
Bottom Line
Regardless of whether you see open source as the enemy or as a complement to commercial software, you stand to gain from learning the secrets to the success of projects that have bested competing teams with a lot more resources. These are true code gladiators whose work bears some examining.
I encourage all developers to look at some of the code out there. Every SourceForge project has a link to the code; and the home pages of the projects listed may have links to their respective discussion boards. Failing that, a Google search or the Wikipedia entry for the application in question will do the trick.
Code on!
Developers whose disdain for open source prevents them from coming into contact with it cheat themselves. This is hardly new knowledge; however, my deductions and learning from open source &0151; in as much as they reflect my unique circumstances and propinquities— may prove useful to you, so I invite you to read on.
Refactoring
People who write commercial software, even those who are the staunchest advocates of refactoring, do not give refactoring its dues. Even in the most refactoring-friendly shops, this is done as a one-off process and never acknowledged as benefitting more than the project at hand.
In the open source world however, refactoring is given more attention because, to paraphrase the famous saying, refactoring anywhere is boost to good code everywhere. My favorite example is how the Linux kernel code has been systematically rewritten even when, as a product, it has consistently outperformed its nearest competitor for a very long time.
So, why do open source developers refactor more? DaVinci-like artistic pride in their work (which is always open for the world to examine) is a prima facie motive.
However, I think there is also a realization among these developers that by improving on earlier work, they improve on their skills as developers and hence their chances of writing better code in the future.
This, I think is a philosophical stance that any development team would profit from. Granted that there are release cycles and deadlines to satisfy, developers can still plan for refactoring. This might be achieved by regularizing schedules and avoiding, in as much as possible, the extremes that make developers go from consecutive all-nighters around release date to WOW-ridden time-killing days.
I know overtime and comp time are sometimes inevitable, but they should be the exception to the rule. A good project manager or head developer should plan for professional development during bench time with as much care as he/she would budget project-earmarked time. Under such circumstances, enhanced refactoring can be worked in quite easily. Google's 20%-time rule is perhaps the most creative way to achieve this.
Code Review
In the commercial software world the very phrase code review has almost the same connotation as administrative leave except that the latter is less tedious. This is very unfortunate especially since it hurts quality and promotes more bugs than any other practice I can imagine.
Agile shops have helped to remove some of the stigma associated with the phrase, but it is the open source world that has led the way. Open source developers welcome code review the way painters prepare to display their work in a swanky gallery. Why is this?
After monitoring discussion threads for couple open source projects, I have to come to realize that the reason for this confidence might stem from the following factors I've observed :
- bad code and its correction is viewed as a learning experience for all as opposed to an embarrassment to the originator
- there is no shame in needing correction and a developer whose code was corrected is still allowed to or encouraged to correct the code of others if necessary
there is a pervasive sense of ownership in and responsibility for "the application" as opposed to one's piece alone- open source developers use their continuous code-review process as a means to collaborate — not recriminate
QA and Testing
Sometimes, inevitably so, despite the best efforts to separate testers from developers, there is pressure on the QA team to certify software too quickly. Understandably, this pressure to ship or release by a certain date is due to the fact that delays cost money.
However, a better practice would be to prioritize QA reports and provide graded certifications so that a product can still ship on time with an assurance given to customers that there will be reasonable number of updates included with the original price of the software product. Besides, updates and downloadable patches are quite inexpensive to provide over the Internet.
One reason many open source products have far fewer bugs than their commercial counterparts is that, in the open source world, bugs are sought feverishly and expected as part of the process. In the commercial world, however, I am afraid bugs are seen as nuisances and departures from "the original plan."
This transparency might seem self-defeating on the surface, imagine how devastating it would be if a disgruntled ex-employee were to leak the truth to the press!
Before anyone mentions Windows Vista, let me hasten to say that I do not condone the release of incomplete products to paying customers. Buyers have a right to expect a certain finish and polish to commercial software products, but they might be more likely to accept certain limitations with the understanding that there are a few minor fixes pending — remember the "we owe" sheet you had to sign at closing when you bought your last home?
Division of Labor
Project managers for both commercial and open source software projects subscribe to the principle of labor division. However, open source projects seem to benefit more from its application. Why is this?
It appears that open source projects are a lot more flexible in how they are structured. In other words, people are willing to drop previously assigned action items and take on new ones as circumstances dictate. Egos seem to be less hypersensitive and players are more flexible.
The reason commercial shops are so inflexible has less to do with IT that with HR. It is a shameful display of incompetence on the part of IT managers that they have not realized that HR titles don't always fit an individual's true capabilities or the specific needs of a project. I would be more forgiving of IT managers if there weren't other fields in which the same issue is handled in a much better way.
In the movie industry, for example, the director may play parts in a scene which, in turn, might be directed by another person. The finished movie is more important than nominal titles and no one feels cheated by such practical reassignments. Likewise, in the software industry, we need to be able to reorganize a team as many times and in as many ways as we need to create a truly best-of-breed product — titles be damned!
Bottom Line
Regardless of whether you see open source as the enemy or as a complement to commercial software, you stand to gain from learning the secrets to the success of projects that have bested competing teams with a lot more resources. These are true code gladiators whose work bears some examining.
I encourage all developers to look at some of the code out there. Every SourceForge project has a link to the code; and the home pages of the projects listed may have links to their respective discussion boards. Failing that, a Google search or the Wikipedia entry for the application in question will do the trick.
Code on!
Subscribe to:
Posts (Atom)