The Without Warranty Wild West Software Industry
Associated blog post date: 18th Mar. 2013, last updated on 28th Mar. 2013, link: http://eklavyasai.blogspot.com/2013/03/the-without-warranty-wild-west-software.html , short link: http://bit.ly/the-without
The world today is pervaded by a huge variety of software from Operating System software to embedded devices software to mobile phone apps. Serious bugs including very scary security flaws in these software are commonplace and accepted as an inescapable reality and risk of using software. Is that a proper stand for the software industry and software customers & users to take?
IMHO, the software industry culture is to blame for the current rather unsure and unprofessional state of the software world. In the nearly three decades that I have seen it, been part of it and experienced it as a user of software, the software industry worldwide seems to operate as a Without Warranty Wild West industry, with current generation of released software seeming to have far more bugs/problems than software on Mainframes/Mini-computers two to three decades ago. Forget application software bugs, the very operating system on which the whole software stack runs is without warranty and has a host of bugs including some terrifying security weaknesses. If you get hit by a malicious virus then it is your bad luck, that's it. You cannot hold the software company accountable. It is this lack of accountability of the software industry that, IMHO, is at the root of the excessive software failures that users have to suffer from.
Once the software industry becomes like other mature engineering industries who are accountable for their products (e.g. a Television set manufacturer is accountable for its product), then, IMHO, the entire software ecosystem including the teaching/academics part of it will be forced to become more "professional" about software development/engineering. But will most software companies, or perhaps any software company, offer warranty for their software easily and of their own accord? I strongly doubt that will happen without outside or public pressure/intervention.
Meanwhile we have to continue to live with the Without Warranty Wild West software world and produce software applications, within time & money constraints, of reasonable quality with some bugs here and there being tolerated like some security weaknesses here and there in the OS itself are tolerated. Is this the ideal software world? Certainly not! But can we freeze application software development till the software world becomes ideal? I don't think so! As far as I know, very, very few people like Prof. David Parnas, http://www.amadon.ca/Public/information.htm, and Prof. Bjarne Stroustrup, http://www.stroustrup.com/, are thinking, writing and talking about these fundamental issues with software development today. And, IMHO, most people in the software industry or software academy don't seem to bother about or, in perhaps the vast majority of cases, even know about, what they are saying!
The result is that, in this far-from-perfect software world, at least for a wide range of application software, as a software developer one also has to join the crowd and try to deliver software quickly, using approaches like component based development, even if the produced software has a few hidden bugs here and there. One may take a lot of care to ensure that one's code is well designed and reasonably tested given the time constraints one is operating under, but one simply cannot control the bugs in the components that one uses for the solution. As an individual, you can't stop or change this "some tolerance for bugs in return for quick and cheap software solution" culture. You, normally, have to go with it. Or else you, as an exception, can simply stay away from providing such solutions. Others then will satisfy the market demand for such solutions by providing them in your place. I could be wrong but that's the way I think it is.
---
I have also added below snippets of my part of a conversation that I had with a software development practitioner on this topic.
I continue to hold the view that, at least for paid/significant cost software, providing a warranty like other fields of manufacturing, will bring order to the chaos that pervades the software industry today. Whether it can be pushed through or is practical enough are different matters. But if you ask the hapless user s/he does not care about whether it is practical enough or not, s/he is fed up of paying significant amount of money for software and having to live with painful bugs in what s/he has paid for!
...
About poor workmen (i.e. poor quality software developers/software engineers) being a problem, I agree. But till the poor quality of output produced by poor workmen is penalized, the market/industry will employ such workmen and make money at the expense of hapless software customers & users. Prof. Parnas and others seem to hold the view that introducing licensure for software engineers will control the poor workmen problem like in other engineering sectors in the Western world. Maybe it will. But despite tremendous efforts of Prof. Parnas and others they, it seems, have not succeeded so far in the Western world adopting licensure for software engineering/development in a big way.
...
I don't want to get into a detailed discussion on the practicality of warranty for software as I have not examined the matter in depth (and neither have the time to do so now). But then let us at least acknowledge that the software industry folks (including me in the past and perhaps in the future as an Open Source developer) are amateurs as compared to professionals from fields like civil engineering or mechanical engineering. Yes, most of us are/were highly paid for our software development work but we are/were only highly paid amateurs. We don't have the right to call ourselves "professionals" as we cannot give any warranty/guarantee about the software solution we provide, in most, if not all, cases.
Please note that warranty does not mean error-free. A Television set may develop faults within its warranty period. Warranty, in my understanding, implies accountability to the extent of replacing a faulty product with a working product at no extra charge to the customer and also compensating a customer for significant damages incurred due to a faulty product.
A distinguished US academic had a small mail exchange with me on this topic. I would like to share my part of that conversation below as I feel it adds some value to the topic.
The point raised was that we all know today's software is poorly engineered. How do I propose to solve the problem?
My response was:
I think software companies need to become accountable by, for example, accepting financial penalties for significant failure of software during a warranty period. Catastrophic failure of software repeatedly should involve a govt. approved industry body examining the software very much like medical bodies examine repeated catastrophic failures of medical doctors. If the industry body deems that the software deployed did not follow minimum practices for design, code, test, etc. then a financial penalty should be levied on the company, and the record of the company and concerned software developers should reflect this failure. This will create a fear of appropriate professional repercussions for poorly engineered software.
As of now, I don't think there is this kind of professional penalty/blot-on-record fear either at the company level or at the individual software developer level. The focus is on getting the minimum stuff done for the customer to sign on the dotted line and make the payment for the customized software developed/product sold. In case of products, the fear is of bad publicity if the product is bad, and so losing out to the competition - that may be a motivator for trying to ensure good quality. In case of customized software solutions, the fear really is of repeat business not coming. IMHO, such fear is nowhere close to the fear that a medical professional has if s/he is involved in negligence or malpractice. We need to bring in this fear of repercussions like in medical practice for badly done software. That may really bring some professional like discipline in software engineering/development.
I hope I am not sounding like some fear-mongering dictator :). IMHO, very unfortunately, without fear of significant repercussions, people, in general, tend to cut corners in almost all walks and activities of life. In Asian countries like India, it is very much the case but I think it applies to quite some extent even in the materially advanced Western countries.
Here's a mail exchange (slightly edited) with another correspondent/friend who kindly permitted me to share it on this blog post.
Friend wrote in response to this blog post: I have one quick observation : When comparing a TV set with a S/w, the manufacturer of a TV will replace the set within warranty period if the operating conditions have not changed.
My (Ravi) response: Well, I guess the warranty will be valid even if operating conditions change but are within the operating conditions for which the warranty is valid.
Friend: Let us say that a TV worked for 3 months and suddenly kicked its bucket. Then he will replace it - however, it should not be because of usage during high / low voltage, pouring water into it, etc.
Ravi: Yes, IMHO, as then the conditions for the warranty would be violated.
Friend: In the case of S/w, is not the scenario different?
Ravi: There certainly will be certain differences between software and an electronics gadget like a TV. But I was going mainly by the principle which I feel can be applied to software too.
Friend: If you have been using the same functionality/features and if the S/w had worked earlier can it really stop working?
Ravi: Yes. The s/w may encounter new data which breaks it (e.g. boundary conditions). Further, nowadays for some software like operating system and anti-virus software you have the automatic updates feature, say over Internet, which may introduce new buggy code into the software, even a few days or weeks after it is installed.
Friend: One scenario is as follows:
For instance, you might use a new choice in a menu and find that it is not doing what it is supposed to do, but perhaps it was not working right from the beginning - just that we did not find out about it.
Ravi: Yes, this is clearly a possibility with today's software which are sometimes overloaded with features/functionality that one does not regularly use.
Friend: For instance, in a TV which claims to have 1000 channels, perhaps we tuned only the first 50 ones and after 6 months when we try to tune the 51st channel we find it is not possible. The above case is similar to this. So if we find it out within the warranty period TV manufacturer replaces it and so should the S/w provider / vendor.
Ravi: Agreed.
Friend: Another scenario is what I wrote first. In a s/w can it happen? If it does, is it problem with H/w or S/w or some other "element" of the system? Whose responsibility should this be?
Ravi: If it is s/w which is the cause e.g. not handling the new data, then the responsibility should be the software vendor's.
Friend: (I may not have) explained myself clearly, but I hope you see there is a difference here? Or is there??!!
Ravi: I am sure there will be differences. S/w warranty will be more complex than a TV warranty.
In this connection, ideally a vendor should publish its QA reports for the software it sells to users. The QA report must detail all the tests that the software was put to. Such clear documentation of the QA process will allow customers to get a feel of the rigor of the software vendor's QA process. If the customer discovers a bug later on s/he can check against the QA report and see how this bug slipped through. The customer will then know whether the QA report missed it or whether it stated that it passed that test. In the latter case, a question would pop up about the truthfulness of the QA report of the vendor.
Such transparent QA reporting will give the customer a feel of how robust the software is, thereby allowing him/her to decide how much to depend on it.
I don't know whether such QA reports are publicly made available for software products now. Maybe open source guys do that. Not sure if a company like --- would do that though :). They will consider it perhaps only if government (i.e. legislators) or courts force them to do it.
... Another mail exchange with same friend ...
Friend: Actually, what I find very very interesting is that so far (a software industry body) or any such body has not thought of this.
Ravi: Perhaps they know about it. (They may not want to consider) any demands for warranty as they may feel that it would negatively impact software companies, which they represent.
Friend: Further, the non-IT MNCs which pay such huge sums also do not seem to be reporting the failure of s/w as a problem - surely it drains them of millions.
If one particular part is faulty, a car manufacturer takes the other party to task, but somehow when we come to s/w everybody seems to have infinite tolerance!!
Ravi: I think the world at large has been dazzled by the achievements of software. That and the huge money power that software companies have, makes these companies very formidable entities to aggressively question. I don't think governments will have the (will) to push software companies hard anywhere in the world today as the world is becoming more and more dependent on software, and (software companies would be having, I guess, significant political lobbying power). It is the Western world justice system that has the (will) to question and even fine software companies - e.g. ---,--- being fined for their practices like restrictive trade practices (---) and for using/capturing data of people from their wireless (home) networks without their permission (---).
I think it is just a matter of time before some software glitch results in a catastrophe, provoking somebody/some entity to take the matter up to a Western world court (I don't know whether it has happened before; maybe it has but as I was not following this space before the past year or two, I do not know of it) - the top Western world judges will not easily buy arguments of software companies that the software is without warranty/"At your own risk" and so they are not legally liable for anything related to their software. They may (apply) some other law.
Friend: Thanks for this really thought-provoking perspective.
Ravi: You are welcome, brother --.
-----
Given below is a comment from blog post associated with above article.
Ravi S. Iyer wrote on March 22, 2013 at 1:25 PM
Even NASA is not able to avoid software glitches. Here's info. about glitch(es) that has/have stalled its latest $2.5 billion (reportedly) Mars Rover Curiosity probe/robot/mission: Time article (archived link: https://web.archive.org/web/20130323015409/http://science.time.com/2013/03/19/mars-rover-curiosity-stands-down-after-new-problem/, short link: http://bit.ly/3c3bcGr ) and Information Week article (archived link: https://web.archive.org/web/20130321094012/http://www.informationweek.com/government/information-management/nasa-curiosity-rover-hit-by-software-sna/240151092, short link: http://bit.ly/32p2RrW ).
A Debate on Warranty for Software
Associated blog post date: 24th Mar. 2013, last updated on 28th Mar. 2013, link: http://eklavyasai.blogspot.com/2013/03/a-debate-on-warranty-for-software.html , short link: http://bit.ly/a-debate
The post, The Without Warranty Wild West Software Industry (previous article in this book), whose contents I mailed out to some correspondents, resulted in a few of them responding with the problems that warranty for software has, and some related views of theirs. I have given some of the responses below, and then my response to them.
Correspondent1 (Corr1) is 'someone with decades of experience in academia and industry'. He wrote:
This is an area I have worked in for decades so it is of great interest.
I think the basic problem you are ignoring is that most software is developed to meet imprecise and incomplete requirements and specifications. I can give you numerous cases where the specifications failed to define all the possible cases that some software needs to cover and left it open to the implementer to decide what to do. There are other cases where assumptions are made but not stated and yet other cases where the specifier just did not know the existence of the case and/or did not know what to do if it occurred.
We are talking about immensely complex systems, far more complex than any machine that is built. Just look at the billions or trillions of states that a large complex system will go through in its lifecycle.
I think that given the limitations of the field today, it is not correct to talk about warranties because a warranty specifies a performance against a specification. However, we are far from having anything like a precise and complete specification for any software system.
Instead, the focus should be on the process used to produce the software: the kind of design, the steps used for its construction and the tests done during the development. All this needs to be recorded and saved for ever.
In the 1980s in the UK, the defence authorities were persuaded to issue standards called DEF 0055 and 0056 for software. The standard had a lot of conditions that no developer could hope to meet (including one that required all tools used during development to themselves have formal proof of their capabilities). The standards have since been made more realistic.
You may be interested to read what Nancy Leveson has written on software and systems reliability (she has written a whole book on the subject), likewise Peter Neumann and others. Their main objection is to the lack of rigorous method during software development, the sort of due diligence that is essential and often watered down or ignored.
Another correspondent (Corr2) wrote that the problem starts at specifications level as software is very poorly specified. He says that Formal Methods can make specifications precise but that industry does not accept it. To see the exact words of his view, please see blog post associated with this article.
My (Ravi's) response to the above:
Yes, I agree with both Corr1 and (Corr2) that it is a very difficult, if not impossible task to define precise specifications for most of the complex software that is developed and used nowadays.
But I still feel a limited warranty of some sort could be considered. Now, I am not an expert on this matter and have not studied the work of researchers on this topic. However, IMHO, I am entitled to express my opinion based on my nearly three decades of experience of the software industry and software academia.
The test cases, test reports and test data that are used by a software vendor to check the quality of the software prior to releasing it to the customer, should be handed over to the customer. This will enable the customer to independently, if the customer so desires, check the quality of the software for themselves. Further, such test information & data can constitute the operating conditions for which the software vendor can provide a warranty. These operating conditions could also specify the version and other details of software components supplied by other vendors that the software solution is based on. E.g. Database, Report Generation tools and the Operating System.
If the customer encounters a bug for a specific use case in the delivered software system, they can check whether the test cases and other operating conditions cover such a use case. If it does cover it and the use case does not work as required in the test environment using the specified/agreed-upon test data, even though the test report claims that it worked, then the customer has a clear case of the software vendor not having tested the software properly as per the agreed software development contract (or software product contract with the variation that the software product company unilaterally fixes the test cases, test data, environment etc.) The limited warranty will come into play with the software vendor having to do whatever is specified in such a warranty, e.g. fixing the bug(s) at no charge to customer, and, in the case of a very-confident/aggressive warranty, provide compensation for loss faced by the customer due to the bug(s).
But will such a limited warranty which applies only to test cases & test data etc. considered at software development time be of any use at all? I think it will. As of now, I don't think that, in typical software deliveries, such test details are given to customers in such a way that they can run the tests independently. Once such a culture gets introduced, customers will get more involved with the testing process. Specifically they may ensure that the test data covers all the cases that they want in the software, as they get a warranty for such test data. If for their test data the software is found to not work during period of warranty they can hold the software vendor accountable! That can give tremendous confidence to customers about some sort of limited accountability for the software, as compared to the situation now where essentially the customer is told that the entire software is to be used "at your own risk"!
Just imagine the business risks that come into play when a business becomes heavily reliant on such software which is used "at your own risk". Further try to imagine a chairperson or managing director of such a business trying to explain to angry shareholders in an AGM, a catastrophic software failure that caused significant damage to the company, and steps that will be taken to avoid similar catastrophic software failures in future.
Now software vendors offering such limited type of warranties may charge extra for the warranty. That is fair, IMHO. The customer gets a choice between less expensive software but without any form of warranty and somewhat more expensive software with some sort of limited warranty. This would parallel many other fields in India. E.g. mobile phones. The customer can buy cheap mobile phones produced by almost unknown company(ies) with no warranty OR mobile phones produced by reputed company(ies) with warranty which are significantly more expensive than the former. There is room in the mobile phone market for both.
Similarly, I think, if some type of limited warranty is offered for some software, over time, we will have both types of software solutions - no warranty and limited warranty, each with their own space in the market.
I should also mention that many devices/machines which use embedded software (e.g. medical devices, some or most cars, airplanes), I presume, are being sold with a warranty for the devices/machines which will include the embedded software part. But that is a specialized use of software, and anyway they seem to be providing a warranty already. It is the only software solution vendors (no device) that I am referring to in this post (and previous post mentioned at the top of this post). Now, in the only software solution space there is a huge variety of software from Operating Systems to Compilers to Database-oriented business applications. While I was writing the above my thoughts have focused more on Database-oriented business applications, as life in Indian towns & cities today seems to be heavily interacting with or dependent on such applications (e.g. banking, mobile top up, railway reservation, hospital admission etc.) but I have tried to be generic. So a lot of it may apply to other kind of software like a mobile spreadsheet application too.
Now about importance to be given to the software process (requirement analysis, requirements capture, design, code, test, deployment etc.) - I entirely agree. Ideally there should be a minimum standard or a set of few minimum standards for the software process which should be published by government approved industry standard bodies. When a software vendor takes up a software development order it should inform the customer about which industry standard process it is following. Further, key artifacts produced as the software is developed using the industry standard process (design specifications, program specifications etc.) should be provided to the customer, so that the customer can, if needed, itself, or by using a third party, inspect the artifacts to check whether the specific industry standard process is indeed being followed.
Like in the limited warranty suggestion, the customer can be given a choice: less expensive software development but which may not follow any industry standard process OR more expensive software development which follows a specific industry standard process and with various artifacts produced by the process being provided to the customer.
An important and significantly large exception to the above is that some software solutions are proprietary to the vendor and so most artifacts produced by any software development process followed for a software solution will have proprietary information that the vendor will not want to share with the customer.
Regarding Formal Methods and (other) techniques to produce bug-free software, my knowledge about these topics is very, very limited. I believe such methods and techniques have not yet been widely adopted by the large majority of software solution providers. Perhaps there are some significant challenges/drawbacks which are impeding their widespread adoption or perhaps there are other political/mind-set type reasons for it. Anyway, their adoption is something that I am not in a position to contribute to in any way. I go by what the mainstream software development industry follows. If and when they switch to Formal Methods in a big way, I may have no option but to consider it seriously then.
A correspondent brought up the "fit for its intended use/purpose" Engineering concept.
Ravi: I did some reading up about it. http://en.wikipedia.org/wiki/Implied_warranty states (for USA):
In common law jurisdictions, an implied warranty is a contract law term for certain assurances that are presumed to be made in the sale of products or real property, due to the circumstances of the sale. These assurances are characterized as warranties irrespective of whether the seller has expressly promised them orally or in writing. They include an implied warranty of fitness for a particular purpose, an implied warranty of merchantability for products, implied warranty of workmanlike quality for services, and an implied warranty of habitability for a home.
...
An implied warranty of fitness for a particular purpose is a warranty implied by law that if a seller knows or has reason to know of a particular purpose for which some item is being purchased by the buyer, the seller is guaranteeing that the item is fit for that particular purpose.[1][2][3]
...
In some jurisdictions, an implied warranty in a sales contract can be expressly disclaimed by the use of specific language, such as the words, "as is" or "with all faults".
[Wiki References:]
1. Primack MA. (2009). Representations, Warranties and Covenants: Back to the Basics in Contracts. National Law Review.
2. "Difference between a Guarantee and Warranty". Archived from the original on December 5, 2016. Retrieved December 5, 2016.
3. Ferara LN, Philips J, Runnicles J. (2007). Some Differences in Law and Practice Between U.K. and U.S. Stock Purchase Agreements. Jones Day Publications.
--- end wiki page extracts ---
So warranties need not always be limited to performance against a specification. A flawed car tyre that leads to many car accidents may not have failed its specifications, but would have failed the "fit for its intended use/purpose" implied warranty and so the manufacturer would be held accountable by the legal system/government.
The software industry seems to have disclaimed the implied warranty of "fit for its intended purpose" by usually adding the words "as is" in the contract.
Okay! So the software industry is legally not accountable even for any implied warranty. But how long can this "unaccountable" thing go on? I think it is just a matter of time before some government or some judiciary somewhere in the world responds to one or more catastrophic software failures badly affecting large number of people, by imposing accountability on software solution providers. And such imposed accountability, in all probability, ain't gonna be pretty!
It would be far, far better if the software industry puts its own house in order by experimenting with some sort of limited accountability. My suggestions outlined earlier may or may not be good enough. There may be better suggestions somewhere else. Or, if nothing good enough is around yet, if the software industry puts its mind to the problem, I am sure they can come up with some concrete suggestions for limited accountability (with its possibly higher costs which the customer must be willing to bear) and which they can experiment with.
Another correspondent questioned my assumptions about a viable market for the more expensive limited-warranty software.
My (Ravi’s) views on it are as follows:
The impression that I have is that people think that software can never have any sort of even limited warranty. As far as I know, concerted efforts by software industry to show that some sort of l