When I was little I remember listening to old people talking about a time when there where no cars, the feeling of excitement and wonder when they saw their firs one, a feeling mixed with a little fear as the mechanical marvel seemed to take over every aspect of life. Where once they played in the road now the car was king, and a ruthless one at that. Communities divided by a constant steam of deadly traffic.
Of course today we take the car for granted. Many have moved away from the workers slums into suburbia and now rely on the car to support this freedom.
We teach our children ‘road sense’ so they can cross the road safely. Most drivers are not deadly speed demons (although in town most people still speed, 40 in a 30 zone IS deadly). Society adjusts and we move on.
Now it seems that its my turn to sound old because I remember a time when there were no PCs.
I remember the excitement of my first Sinclair ZX80, the awe of seeing the colour ZX Spectrum.
But now I feel the fear.
Now don’t get me wrong here, I am a great believer in the usefulness of computers, I have a degree in computer systems engineering, I have made a career out of devising and tweaking computer control systems for cars.
But still, now I feel the fear.
When I was studying to become and engineer, every step of the way I was told of the importance of doing things properly. With a large computer program one has to exactly and correctly specify what it should do in every detail. One must also specify what it must not do! Once the program is written then it must be tested against this specification and every possible combination of circumstances must be tested. That way there are no ‘bugs’ and unexpected effects.
But life is not like that.
The software (and also hardware now) on almost everything is so complex that it requires a computer program just to be able to test it.
No one programmer can do the whole thing, its just too big, so we have teams. So now we have programs to help the teams work together without bits getting left out and prevent miss interpretations etc.
But we live in a capitalist society. Its not just the engineers that create products, its corporations. Many individuals with their own beliefs on how things should be done dictating the boundaries and detail of what the engineer can do but without a sound understanding of the technicalities.
Money has too be made (exceptions include Linux (three cheers)) and so whole chunks of code from other programs are grafted in to new programs, the people producing this new program may not know the details of how this chunk was written and all its effects. Sometimes there may be a ‘surprise’ effect caused by the interaction of this chunk with the rest of the program, other chunks grafted in or indeed other programs running on the same machine or network.
Testing takes time and money and delays the launch date. Some things just cant be tested completely due to their nature, for example if your program predicts the weather then how do you test every possible combination of weather across the whole world and still meet the deadlines.
Also the hardware too is so complex that it is not commercially viable to test everything, or indeed possible. With several million transistors on a single chip is never going to get tested for the effects of every combination of individual transistor failures.
So that’s where we are today. Our systems are only partially tested and often a patchwork of other peoples work all stuck together with hope and optimism. Or indeed sometimes cynicism.
Many consumer products are made by inexperienced teams and pushed out by unscrupulous corporations (particularly in countries where software standards are not enforced) and are largely unproven.
Many of us have experienced the result of this growing problem, such as the PC just locking up when you try a new program or simply getting slower and slower as time goes by. These bug and software faults are so common that many people think it is normal for computers to behave like this. For instance the PC I am writing this on is twelve years old, it still does everything it was designed to and since running Linux it hasn’t slowed right down or ground to a halt, yet still most people accept that computers need replacing every other year and expect it to slow down over time. It must be realised that it doesn’t have to be this way, technically, but commercial pressures will continue to make the problem worse and this will be compounded as more and more code is piled on to bring use ever more features.
Complexity is a big problem and is the subject of many a professors career, things are getting more and more complex and there is no proper engineering control on it.
Now, the reason that I am writing this is not just to have a good whinge about my computer crashing or indeed to complain about commercial forces ruining good engineering. Those things make me angry, but they are not the cause of my fear.
The fear stems from how we are using these systems as a society, how we are relying on the unreliable.
Computer systems are now increasingly being used as part of the law enforcement system, finance control, travel systems and even food production
Speed cameras always cause a good argument so I will stir thing up a bit further. Now I know very well that excessive speed increases danger of injury and general twisting of machinery and putting a speed camera outside a school is no bad thing.
The issue for me comes from the fact that the picture generates an automatic fine for a person. There is no human judgement in the loop, bang, guilty until proven innocent. And that’s wrong.
A friend of mine suffered from a theft from his car, not the usual sort of theft, the number plates were stolen. It turns out that persons of criminal persuasion are stealing a car then cruising round till they find an identical type of car and putting those plate on theirs. Then they can generate speeding fines and parking tickets with impunity and even commit serious crime knowing full well that the system will point the finger at some one else. It even cause the police to waste time with the wrong chap, keeping the heat off the criminals long enough for them to make their escape.
Guilty until proven innocent, trial by computer, not good, not very British.
Maybe soon we will all have ID cards. This means that criminals only need to forge one item instead of a string off items as at present, thus making their life easier. The systems used for security are simply to complex to be testable, and driven down on price so the quality is marginal. Its simply not reliable.
If you want quality you have to pay for it because quality systems take more time to engineer and more time to test and it all costs money.
We are entering the beginning of a time when cars become more autonomous, adaptive cruise control will adjust the car speed to the traffic conditions, lane assist can nudge the steering to stop you drifting off your chosen path, we even have auto parking systems. It is a logical step to bring all these ideas together and link them to the sat nav to create fully autonomous cars, Google are investing heavily in this idea. Once the systems become common there will be increasing pressure to ban manual driving, after all an autonomous car doesn’t get road rage, doesn’t speed, can see through fog, never gets distracted and should never crash. All those computer systems running all those programs written by thousands of different people at different times in different places and controlling your car….
In the near future there will be an attempt to make remote vehicle arrestors mandatory on all new cars. This system uses ABS systems that have full authority breaking and engine management systems to bring a car to a halt using a radio command that only police will have. In a simplistic world this is great, you report your car stolen and the police can bring it to a halt when the conditions are safe. No more getaway cars. Well, unless criminals use older cars, but that loophole is easily solved by making classic cars illegal and crushing them all!
The problems include accidental stopping of the car (you cant prove the software completely due to its complexity and you cant prove the hardware completely because you cant test every failure and every type of possible radio interference etc), incorrect use by the police or other agencies, vehicle being stopped by criminals equipped with illicit stopping systems for the purpose of car jacking. Finally there is always a way to bypass the system, always a loop hole, a bug, a back door or an ‘unintentional feature’.
I was on a train in Germany last year which suddenly stopped in the middle of no where without warning, brakes full on. Luckily I had finished my coffee so the cup was empty when it slid of the table. The cause of this potentially dangerous emergency stop was a software error in the very system that is supposed to protect the train from crashes.
Our corporate based society does not allow for well written systems to be made as profitably as the quickly written ones.
This is a real problem and is getting worse as more systems are used.
In my life I rely on a mobile phone, I rely on my car, my computer, email, bank direct debits, automatic payments, alarm clock, microwave, fridge, washing machine, traffic lights etc. The power feeding my home is controlled by systems all linked together in a network. The amount of chlorine in the water I drink is monitored electronically. Aeroplanes are flown expertly by computers over my head, the air traffic is controlled by other computers.
I use my switch card to pay for car tax, the little computer in the post office reads my details and talks to one of many networked computers at the bank, the figure in my account file is reduced and a message sent to the post office bank computer to tell it to increase the number in its account. Then a message is sent to a computer at DVLA and it changes the value of a variable in a file so that when another program does its daily check of who has tax it will not automatically send a message to another computer to send me a fine and automatically turn me into a criminal. I never see these computers and they never see me. But they can bankrupt me accidentally or send me to jail.
These systems are not designed completely by engineers, the specifications and design constraints are created by politicians and computer sales executives who simply don’t understand.
When I was a child, I was proud to be British, a country that believed in tolerance, understanding and fair play. I was proud of my country.
Now I am scared of my country and the automatic systems that rule my life.
My bank local branch has just got rid of all its cashiers, you have to use the machines now. Signatures have been replaced with PINs.
Make no mistake, these systems give us great ability as a society and as in individual. The principles of the systems are very good, it’s often empowering and can change lives for the better. Even this blog site gives me a platform to express my beliefs and concerns in a way that was impossible a generation ago. I am a great believer in technology.
But as far as I can see if we are to rely on systems then they must be reliable.
Also, there must always be a human in the loop when ever civil liberty is at stake.
And finally, there must always be a manual back up for those odd days when thing don’t quite work the way they should.