Richard Lary,
DIGITAL Storage Technical Director

Welcome to the winter 1997-98 issue of the DIGITAL Technical Journal. This issue does not have a single theme; it contains a potpourri of papers on a wide range of technical topics. This provides the foreword writer with a small gift and a not-so-small headache.

The gift is the opportunity to tout the continuing fecundity of DIGITAL’s engineering community. All the papers in this issue of the Journal come from product development groups in DIGITAL, and all the technology described herein is directly applicable to the problems of using computers in the real world. The papers themselves cover a wide range of topics: designing storage buses and their infrastructure; building IP routers that reduce network delays caused by link or router failure; sharing 3-D graphical and audio data across networks of computers with different windowing systems; and debugging programs written in languages that incorporate data parallelism.

The headache, of course, stems from this very diversity. Any attempt to derive some set of common underlying principles other than "make better stuff" from this collection is doomed to sophistry. And my technical background is too narrow to provide any significant embellishment to any of the papers outside the domain of storage systems. So, with apologies to the other authors, I am forced to restrict my comments to what I know - the background and impact of Bill Ham’s work on advances in parallel SCSI which are presented in his paper in this Journal.

Bill Ham’s paper not only describes a significant technical achievement; it illustrates DIGITAL’s shift from engineering proprietary storage systems to engineering open storage systems.

The SCSI bus was developed during the early 1980s as one of many attempts to standardize the interface to storage devices. It succeeded beyond the expectations of its developers, largely because it supported a device model that was abstract enough to be extensible but inexpensive enough to be implemented in the technology of the time. For all its advantages, however, SCSI suffered from poor engineering at the physical level. This was a direct result of the way it was developed. The diverse corporate representatives that defined SCSI did not have the time or money to specify and build custom bus infrastructure components (transceivers, cables, terminators, etc.), so they used commonly available parts. A lack of sophistication in specifying physical interface parameters resulted in a specification that allowed too much component variation. As a result, it was difficult to build reliable, multi-box systems using SCSI.

DIGITAL’s attitude towards SCSI during this period was to ignore it and hope it would go away. We had designed our own proprietary Digital Storage Architecture (DSA), which utilized an abstract and extensible device model and also incorporated many large system features, including a robust physical interconnect. We controlled the design and manufacture of all DSA components and could thus guarantee that they all met tight architectural specifications. Moreover, DSA was a key enabling technology for VMS Clusters, the individual DSA components were competitive with their counterparts from the proprietary storage architectures of other large systems companies, our customers were happy, and the storage business was profitable. We were feeling quite pleased with ourselves - and we were profoundly ignorant of the power of a successful open market standard, since one had never existed in the storage world.

During the latter half of the 1980s, SCSI grew steadily in popularity until it dominated the workstation and small-server markets. These systems had at most a few disk drives on them, and SCSI’s signal integrity problems were manageable in that context. They were not manageable in the larger and more demanding data center systems, and so SCSI was not used there. The SCSI standards group was aware of the bus’s deficiencies, however, and as the decade progressed, the group made amendments to the standard to eliminate many of them. By the turn of the decade, several independent subsystem vendors were selling subsystems utilizing SCSI devices as storage for large DIGITAL systems. These subsystems did not, in general, have the features, performance, or robustness of our subsystems, but they were significantly cheaper and improving all the time. By 1991, it had become obvious to us that we would not be able to compete with these systems in the long run. They were leveraging an entire industry’s investment and talent and were reaping the cost benefits of high-volume manufacturing; whereas we had to design and manufacture (at relatively low volume) every component of every DSA system ourselves.

Our position was untenable. We had to change our strategy and embrace the bus that we had so studiously ignored.

We designed a modular packaging architecture for SCSI devices (known commercially as StorageWorks) and a set of storage array controllers that interfaced these devices to our systems (and systems from other major vendors as well). We also became active participants in the SCSI standards process. Where DIGITAL had previously sent one or two engineers to SCSI standards meetings strictly to gather information, we started to send up to half-a-dozen engineers to listen, learn, participate in debate, help with the grunt work of the standards process, and make proposals to amend or extend the standard in directions useful to us and our customers.

Our new modular packaging design allowed our customers to install and remove storage devices themselves and to migrate storage devices between systems, even between systems built by different system vendors. This modularity proved to be a very valuable feature to our customers. However, it required us to build a physical infrastructure for the SCSI bus that had the robustness needed by our large systems and that could accommodate a great deal of variability in configuration, and to use a bus that was known to have residual signal integrity problems in its physical interconnect. We were understandably worried about this, worried enough to charter a small group of engineers as a SCSI Bus Technical Office (SBTO) within the storage group, and to develop short-term configuration guidelines for our packaging architecture and long-term technical proposals for the SCSI physical bus architecture. Bill Ham has been the head of SBTO since its inception and has also been our representative to the SCSI committee on all matters relating to the physical bus interconnect.

In the summer of 1993, Bill completed a study of the signal integrity issues surrounding parallel SCSI. His conclusions were startling. The SCSI standards committee had, over the years, made enough improvements in the basic transmission line characteristics of the SCSI bus that most of the remaining signal integrity problems were due to the variations in component parameters allowed by the SCSI specification. Exercising tighter control over component variation - through building selected components or through purchase specifications with our suppliers - would not only produce excellent signal integrity in our packaging but would allow the maximum clock rate of the bus to be doubled while maintaining excellent signal integrity and backwards compatibility with existing SCSI devices. Bill’s results also indicated that the maximum clock rate could be increased even further, with more work.

This discovery came at a critical time in the evolution of the SCSI standard. Much of the SCSI standard committee’s effort in the early part of the 1990s was being spent in modifying the SCSI standard so that serial buses could carry the higher level SCSI bus protocols. The committee had started this work under the assumption that parallel SCSI was "out of gas" in performance, and the new serial bus variants would supplant it by mid-decade. However, by 1993 not only was the definition and implementation of the serial bus going slower than expected, but there were three independent and incompatible serial bus proposals, each with unique useful features and unique drawbacks, each with a cadre of supporters among the industry representatives. The market would ultimately choose which serial buses would thrive; but it was highly unlikely that all three would thrive. Storage vendors that made the wrong bus choice would suffer for it. Most galling to the technophiles among us, the market’s choice could not be predicted from the technical merits of the contenders. If it could, we’d all have Betamax VCRs in our homes today.

So, DIGITAL decided to have Bill present his results to the SCSI committee at its November 1993 meeting and recommend that the committee extend the SCSI specification to allow the bus to run at up to twice its old maximum clock rate if the components in the physical interconnect met the tighter specifications. Our motive in doing this was purely selfish: we were not ready to choose among the serial bus proposals, yet we would soon need more performance than parallel SCSI could offer. A higher performance parallel SCSI would allow us to improve our storage subsystem performance without having to stake our fortunes on a potential Betamax.

Bill’s presentation at the SCSI committee meeting was met with enthusiastic approval. It turned out - surprise! - that other system vendors were feeling as uneasy as we were about the serial SCSI buses. The proposal, christened UltraSCSI, was adopted as an extension to the parallel SCSI standard. Bill Ham and the SBTO then worked with component vendors and the SCSI committee to develop the thinner cables, smaller connectors, and SCSI expander circuits described in his paper, all with the aim of keeping parallel SCSI as a desirable alternative to the serial SCSI buses. Today, four years after its committee debut, UltraSCSI is solidly entrenched in the storage market. In fact, storage market analysts are now projecting that the combined volume of devices on all serial SCSI buses (yes, there are still three, but the market has already picked one, Fibre Channel, as the winner) will not exceed parallel SCSI device volumes until early in the next century. And the SCSI committee has finished extending the parallel SCSI specification to achieve a second doubling of maximum bus clock and is in the midst of defining a third doubling.

Without hyperbole it can be said that the technology embodied in Bill Ham’s paper has directly affected the course of the computer storage industry, and it continues to affect positively DIGITAL’s position in that industry. Enjoy reading the paper and those that follow it in this issue.