7-Sep-86 21:19:55-PDT,14188;000000000000 Mail-From: NEUMANN created at 7-Sep-86 21:16:17 Date: Sun 7 Sep 86 21:16:17-PDT From: RISKS FORUM (Peter G. Neumann -- Coordinator) Subject: RISKS-3.51 Sender: NEUMANN@CSL.SRI.COM To: RISKS-LIST@CSL.SRI.COM RISKS-LIST: RISKS-FORUM Digest, Sunday, 7 September 1986 Volume 3 : Issue 51 FORUM ON RISKS TO THE PUBLIC IN COMPUTER SYSTEMS ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator Contents: Computer almost created swing vote (Bjorn Freeman-Benson) Computer Sabotage of Encyclopedia Brittania (Rosanna Lee) F-16 software (Wayne Throop) Arbiter failures and design failures (Martin Harriman) Systems errors (hardware AND humans) (Bill Janssen) Re: Terminal (!) lockup (Roy Smith) The RISKS Forum is moderated. Contributions should be relevant, sound, in good taste, objective, coherent, concise, nonrepetitious. Diversity is welcome. (Contributions to RISKS@CSL.SRI.COM, Requests to RISKS-Request@CSL.SRI.COM) (Back issues Vol i Issue j available in CSL.SRI.COM:RISKS-i.j. Summary Contents in MAXj for each i; Vol 1: RISKS-1.46; Vol 2: RISKS-2.57.) ---------------------------------------------------------------------- Date: Sun, 7 Sep 86 10:44:01 PDT From: bnfb@uw-june.arpa (Bjorn Freeman-Benson) To: risks@CSL.SRI.COM Subject: Computer almost created swing vote Quoted without permission from the Seattle Times, Sunday Sep. 7, 1986: AP, PHOENIX, Ariz. -- Tuesday's primary elections in Maricopa County would have been a mess if officials hadn't figured out that a compuer was set up to give all Republican votes to the Democrats and vice versa. "If it had gone undetected, there would have a major, major problem with the election," County Recorder Keith Poletis said Friday. Poletis said that, if the compuer hadn't been fixed, a race with three Republicans and one Democrat would given the Democrat's votes to one of the Rebulicans. Votes cast for the remaining Republicans would have been zapped into the void by the computer, because the software would find no Democratic opponents. "The computer sorts the cards into two piles, and it was sorting the Republicans into the Democrats' slots and the Democrats into the Republican slots," Poletis said. A clerical error made the computer's cards were ordered was to blame for the mix-up, said Joe Martina, director of the county's computer systems. The error was caught during the secretary of state's test of the country's cards late Thursday. End quote. In my mind, this brings up an interesting question: should errors like this be reported (1) to the general public and (2) to the software engineering community? I think the answer to (2) is yes -- the more data we have on the types of errors that occur involving computers, the better grasp we will have on solving them. However, for (1), I see this arguement: Con - The testing procedures before acceptance caught the error. - The public will just lose faith in computers. Pro - The public should know, because what if the testing hadn't? Con - The public in general is not knowledgeable about computers and software, and the general press is sensationalist. Thus any case reported will not be studied in the necessary depth. - An analogy with civil engineering: should the public know that the first design for a bridge collapsed during testing? Or is it just enough to know that the final bridge works? Bjorn N Freeman-Benson ------------------------------ Date: Sat 6 Sep 86 18:09:51-PDT From: Rosanna Lee Subject: Computer Sabotage of Encyclopedia Brittania To: risks@CSL.SRI.COM Chicago Tribune [From San Jose Mercury News, Friday, Sept 5, 1986] LAID-OFF WORKER SABOTAGES ENCYCLOPEDIA CHICAGO - An employee of the Encyclopedia Britannica, disgruntled at having been laid off, apparently committed computer sabotage by altering portions of the text being prepared for updated editions of the renowned encyclopedia. The discovery has prompted an exhaustive check to ensure that painstaking work, in the words of the editor, "is not turned into garbage." "We have uncovered evidence of deliberate sabotage in the EB computer files," editor-in-chief Tom Goetz disclosed in an Aug. 28 memo to editorial personnel at the chicago headquarters of the oldest continually published reference work in the English language. The unidentified former employee has confessed and is helping undo the damage, a spokesman said, although the company may press criminal charges. He said the 44-million word 1987 edition is safe, but employees are believed to be laboring overtime to catch alterations that could find their way into the 1988 edition. Among the former employee's more vivid changes, sources said, was changing references to Jesus Christ to Allah, the Moslem name for God. Goetz declined to comment Thursday other than to say, "Everything is under control." Another industry executive said, "In the computer age, this is exactly what we have nightmares about." In the first of three memos to editorial staffers, Goetz wrote, "What is perhaps most distressing for each of us is the knowledge that some of our hard work has been turned into garbage by either a very sick or a very vicious person." At the time, he said that the actions constituted a crime under Illinois law, that the company planned to pursue legal actions "vigorously" and that it was issuing new computer passwords to employees. In a staff memo dated Wednesday, Goetz informed employees that "we have successfully concluded the matter of the sabotage of the encyclopedia's data base. "The 1987 printing is secure," Goetz stated. The publication first was alerted to a problem, sources said, when a worker scanned the computer data base and discovered the clearly odd insertion of the names of a company executive and a private consulting firm apparently [There are several problems in believing that this audit-trail approach is fool-proof. First of all, it relies on a password. Masquerading is therefore a concern. The second is probably more important -- any self-respecting system programmer or cracker is probably able to alter the audit trail. It is dangerous to assume that the only disgruntled employess are those who are NOT computer sophisticates... PGN] ------------------------------ Date: Fri, 5 Sep 86 13:19:25 edt From: rti-sel!dg_rtp!throopw%mcnc.csnet@CSNET-RELAY.ARPA Subject: F-16 software Apparently-To: mcnc!csl.sri.com!risks > It sounds very funny that the software would let you drop a bomb on the wing > while in inverted flight but is it really important to prevent this? Is it > worth the chance of introducing a new bug to fix this very minor problem? > [The probability is clearly NONZERO. It is very dangerous to start > making assumptions in programming about being able to leave out an > exception condition simply because you think it cannot arise. Such > assumptions have a nasty habit of interacting with other assumptions > or propagating. PGN] It is also dangerous to start making assumptions about the ways in which the system will be used. Can you really not think of a reason why one would want to "drop" a bomb while the dorsal surface of the plane points towards the planet's center (a possible interpretation of "inverted")? I can think of several. I am trying to make the point that the gross simplification of "preventing bomb release while inverted" doesn't map very well to what I assume the actual goal is: "preventing weapons discharge from damaging the aircraft". This is yet another instance where the assumptions made to simplify a real-world situation to manageable size can easily lead to design "errors", and is an architypical "computer risk" in the use of relatively simple computer models of reality. In addition to all this, it may well be that one doesn't *want* to prevent all possible modes weapons discharge that may damage the aircraft... some of them may be useful techniques for use in extreme situations. The more control, The more that requires control. This is the road to chaos. --- PanSpechi aphorism {Frank Herbert} Wayne Throop !mcnc!rti-sel!dg_rtp!throopw ------------------------------ Date: Fri, 5 Sep 86 09:38 PDT From: Martin Harriman To: risks@CSL.SRI.COM Subject: Arbiter failures and design failures Bob Estell raises two quite different failure mechanisms in his message. The first mechanism he mentions is the well known problem of making a reliable arbiter; he then goes on to discuss the quite different problem of design errors in hardware, microcode, or systems software. The arbiter problem is well known; fundamentally, there is no absolutely reliable way to sample asynchronous signals in a synchronous system, though there are ways of greatly reducing the probability of failure. In this sense, no computer which incorporates asynchronous interrupts is deterministic, since you can not predict its behavior cycle by cycle. It is important to take this effect into account in the design of the system and any software which cares about the timing of these external events. There are other interesting failure modes, where the arbiter essentially says "maybe," instead of giving a clear yes or no answer; careful circuit design can reduce the probability of these failures to one failure every few thousand years (at least according to our last set of simulations). The bulk of Bob's message is a discussion of the probability of design bugs. Anyone who has seen the errata sheets for a microprocessor or the ECO history of a mainframe will know that computer hardware is imperfect. This may be news to some computer programmers, but there is such a thing as a computer error; for instance, the first stepping of Intel's 80186 was convinced that the product of two negative numbers was negative. --Martin Harriman (martin%srucad@sc.intel.com) Intel Santa Cruz ------------------------------ Date: Fri, 5 Sep 86 18:07:08 CDT From: Bill Janssen To: risks@sri-csl.ARPA Subject: Systems errors (hardware AND humans) Bob Estell's note on machine errors made me think of an error that I found some years ago. I was writing a C program that, among other things, provided a virtual connection between two serial ports. The code looked something like this: while (in_connection_mode) { if (input_available(port1)) port2->output = port1->input; else if (input_available(port2)) port1->output = port2->input; } where `port1' and `port2' were pointers to register banks on the serial port controllers. When we tried it out, it didn't work. To make a long story short, it didn't work because assembly code for "port2->output = port1->input" was produced very efficiently as (something like) "MOVB 4(A4),8(A5)", which would still have been OK, except that both serial ports were on the same chip and the chip needed a recovery interval after doing a read before doing a write. Working code used the line "port2->output = temp = port1->input", to introduce a slight delay!! Now, where's the source of the error here? What bugs me is that you can PROVE that the (non-functional) code functions properly... if you ignore the hardware quirks, which aren't documented. And if the compiler produced less efficient code (load register; store register) the HLL code would work. And if the machine architecture didn't have memory-to-memory move instructions the code would work. And if the computer clock was slower, the code would work. I tend to think that the error was in the characterization of the hardware, which described the two serial ports as independent. But perhaps the error is actually in not VERIFYING the hardware characterization... Bill -- Bill Janssen, MCC Software Technology 9430 Research Blvd, Austin, Texas 78759 ARPA: janssen@mcc.com PHONE: (512) 339-3682 UUCP: {ihnp4,seismo,harvard,gatech,pyramid}!ut-sally!im4u!milano!janssen ------------------------------ Date: Fri, 5 Sep 86 18:23:34 edt From: cmcl2!phri!roy@seismo.CSS.GOV (Roy Smith) To: cmcl2!seismo!csl.sri.com!risks@seismo.CSS.GOV Subject: Re: Terminal (!) lockup I wonder: How common is this property [being able to break it by pushing the wrong combination of buttons] of terminals (or other equipment)? We have some CTS-2400 auto-dial modems that let you set all sorts of parameters that get stored in eeprom. It's not too hard to set it up so it doesn't echo and doesn't produce any output at all. This condition persists even after power-cycling. It's not really dead, but unless you realize what you did and know the magic sequence to turn back on echoing and command processing, it sure looks that way (if it looks like a duck and quacks like a duck ...) Take a typical time-sharing system, erase the boot block from disk and turn it off. You've sure done a nice imitation of breaking it (I consider having to toggle in a binary boot program as very much akin to opening up a terminal to fix it). If you've got a writeable control store, you could mess yourself up even more. The (clever) people who designed the Apple LaserWriter must have been thinking along these lines. There are 2 serial interfaces on the LW. You can run a little PostScript to change the baud rate (stored in eeprom) on either or both. If you want to disable one interface, you just set its baud rate to 0. According to the documention (I've never tried it :-)) it won't let you set both channels to 0 baud. If you could, there would be no way to talk to it short of yanking the eeprom. ------------------------------ End of RISKS-FORUM Digest ************************ -------