HITCHHIKER'S GUIDE TO RSX
An Introduction to RSX for Business-Application Developers
Revision 0.0
Disclaimer
This is a preliminary version and is not quite one-hundred
percent complete. Please send comments, questions, and
recommendations -- on this document -- to A-to-Z Base Product
Marketing, MKO1-2/E25, Digital Equipment Corporation, Continental
Boulevard, Merrimack, New Hampshire 03054.
The information in this document is subject to change without
notice and should not be construed as a commitment by Digital
Equipment Corporation. Digital Equipment Corporation assumes no
responsibility for any errors that may appear in this document.
The software described in this document is furnished under a
license and may only be used or copied in accordance with the
terms of such license.
No responsibility is assumed for the use or reliability of
software on equipment that is not supplied by Digital Equipment
Corporation or its affiliated companies.
Copyright (C) 1986 by Digital Equipment Corporation
All Rights Reserved.
Printed in U.S.A.
September, 1986
Hitchhiker's Guide to RSX
Revision 0.0
Table of Contents
Chapter 1 Introduction
1.1 Purpose of this Guide
1.2 Intended Audience
1.3 On the Synergy of Operating Systems and
Applications
1.4 Elements of Application Tuning
Chapter 2 Overview of RSX
2.1 History of RSX
2.2 RSX Version 3.0
Chapter 3 Getting Started
3.1 The Three Faces of RSX
3.1.1 Micro/RSX
3.1.2 Pregenned M-PLUS
3.1.3 RSX-11M-PLUS
3.2 Installation and Startup
3.2.1 Installing Micro/RSX
3.2.2 Installing Pregenerated M-PLUS
3.2.3 Installing M-PLUS
3.3 Logging in and other First Day Tasks
3.3.1 Logging into the Prebuilt Accounts
3.3.2 Post Installation Cleanup
3.3.3 Customizing Startup
3.3.4 Configuring Terminals and Printers
3.3.5 Creating User Accounts
3.4 Installing Layered Products
3.4.1 Installation on Micro/RSX
3.4.2 Installation on M-PLUS
Chapter 4 Terminals
4.1 Cost of I/O
4.2 Specific Cost of Terminal I/O
4.2.1 Interrupt Service
4.2.2 Scheduler Overhead
4.3 Minimizing Costs
4.3.1 Blocking Output Requests
4.3.2 Blocking Terminal Input
4.3.3 Using DMA Terminal Hardware
Chapter 5 Record Management Services (RMS)
5.1 Introduction to RMS
5.2 File Organizations
5.2.1 Sequential
5.2.2 Relative
5.2.3 Indexed
5.2.3.1 Structure
5.2.3.2 Population
5.2.3.3 Index Activity
5.2.3.4 Storage Overhead
5.2.3.5 Interlocks
5.3 File Design
5.3.1 Which Files to Design
5.3.2 Selecting an Organization
5.3.3 Common Design Factors
5.3.3.1 Allocation
5.3.3.2 Extend Quantity
5.3.3.3 Contiguity
5.3.3.4 Carriage Control
5.3.4 Designing Sequential Files
5.3.5 Designing Relative Files
5.3.6 Designing Indexed Files
5.3.6.1 Specifying Key Fields
5.3.6.2 Sizing Buckets
5.3.6.3 Areas
5.4 Tuning RMS Files
5.4.1 Avoid File Opens
5.4.2 Use the Simplest Possible File Structure
5.4.3 Beware of Overloading Directories
5.4.4 Pre-Allocate the File
5.4.5 Make Critical Files Contiguous
5.4.6 Substitute Code for Mechanical Movement
5.4.7 Distribute the Load
5.4.8 Avoid Going Overboard
Chapter 6 RMS Utilities
6.1 RMSDES File Design Utility
6.2 RMSIFL Indexed File Load Utility
6.3 RMSCNV File Conversion Utility
Chapter 7 RMS for CTS-300 Users
7.1 Record Locks
7.2 Access Modes
7.3 Interchanging Sequential and Relative Files
7.4 Extending Files
7.5 Print Files
Chapter 8 Flow Control
8.1 Concept of a Task
8.1.1 Installing a Task
8.1.2 Task Names
8.2 Requesting an Installed Task
8.2.1 RUN a Task
8.2.1.1 Running an Installed Task
8.2.1.2 Running a Task from a File
8.2.1.3 Running a System Utility
8.2.1.4 RUN as a Scheduling Command
8.2.2 Chaining between Tasks
8.2.2.1 Indirect Chaining
8.2.2.2 Direct Chaining
8.2.2.3 Chaining to a Command File
8.2.3 Spawning an Offspring Task
8.2.3.1 Indirect Spawn
8.2.3.2 Direct Spawn
8.3 Parent-Offspring
8.3.1 Command Line
8.3.2 Status Code
8.4 Intertask Communication
8.4.1 Send/Receive
8.4.2 Global Event Flags
Chapter 9 Linking and Overlays
9.1 Invoking TKB
9.2 Disk Overlays
9.2.1 Trees
9.2.2 Co-Trees
9.3 Memory Resident Overlays
9.4 Guidelines for Use
9.5 Linking to RMS
9.5.1 Disk Resident RMS
9.5.2 Memory Resident RMS
9.5.3 Clustered RMS
9.5.4 Supervisor Mode RMS
9.6 Useful Options
9.6.1 Privilege
9.6.2 Task Priority
9.6.3 Task Name
9.6.4 Cross Reference Map
Chapter 10 Packaging Applications for RSX
10.1 Kitting for Micro/RSX
10.1.1 Micro/RSX Layered Product Installation
Architecture
10.1.2 Creating the Kit
10.1.3 Developer's Note
10.2 Kitting for M-PLUS
10.3 Kitting Applications for A-to-Z
Chapter 11 System Operation
11.1 Terminals
11.1.1 CLI
11.1.2 BROADCAST
11.1.3 CONTROL=C
11.1.4 INQUIRE
11.1.5 FULL_DUPLEX
11.1.6 ESCAPE
11.1.7 EIGHTBIT
11.1.8 HOSTSYNCH
11.1.9 SERIAL
11.1.10 SLAVE
11.1.11 LOWERCASE
11.1.12 WRAP
11.1.13 TYPEAHEAD
11.2 Partitioning Memory
11.2.1 Exec and Primary Pool
11.2.2 Secondary Pool and Extension
11.2.3 Task Space
11.2.4 Cache Region
11.3 Disks
11.3.1 Initializing
11.3.1.1 Preparing a Disk
11.3.1.2 Initializing a Volume
11.3.2 Mounting
11.3.3 Dismounting
11.4 Creating Directories
11.5 Files
11.5.1 File Performance
11.5.2 File Protection
11.6 Logical Names
11.7 Disk Caching
11.8 Printers and Queues
11.8.1 Queue Mechanism
11.8.2 Submitting Jobs
11.8.3 Startup
11.8.4 Shutdown
11.8.5 Transparent Spooling
Chapter 12 Troubleshooting
12.1 Before You Start
12.2 Diagnostic Tools
12.2.1 Resource Monitor Display (RMD)
12.2.1.1 Memory
12.2.1.2 Active Task Display
12.2.1.3 Task Display
12.2.1.4 I/O Counts Display
12.2.1.5 System Statistics
12.2.1.6 Cache Region Display
12.2.1.7 Detailed Cache Statistics
12.2.2 Error Logging
12.2.3 Resource Accounting
12.3 What to Look For
12.3.1 Disk Saturation
12.3.2 Insufficient Memory
12.3.3 Unbalanced Load
12.3.4 Insufficient Pool
12.3.5 File Contention
12.3.6 Low Cache Hit Rates
12.3.7 Insufficient Checkpoint Space
12.3.8 Excessive File Opens
12.3.9 Overloaded Directories
12.3.10 Jobs at Wrong Priority
12.3.11 Misuse of Batch
Hitchhiker's Guide to RSX
D O N ' T P A N I C
Revision 0.0
Chapter 1
Introduction
This guide introduces the RSX operating system to the Small
Business Application Developer. It contains information about
those aspects of RSX which are of particular interest and
importance to application developers who are considering or are
in the process of migrating commercial applications to RSX or
creating new ones. Much of the information will also be of use
to developers considering a move to VAX/VMS since the two systems
share many characteristics.
The guide describes general principles for developing a new
application or adapting an existing one to RSX. It also covers
certain areas of special importance to small business application
designers. RSX is a mature operating system and there are many
case histories of successful use in a small business environment.
Unfortunately, there have also been instances in which developers
have encountered difficulty in making the transition from simpler
environments. In many cases the effort required to correct the
problems proved to be trivial. All that was lacking was
information. There is nothing about RSX which makes it
unsuitable as a base for small business software. Developers who
are able to invest the time and energy in developing an
understanding of the strengths and weaknesses of RSX will
maximize their chances at a smooth, successful and inexpensive
conversion.
1.1 The Purpose of this Guide.
Operating systems are all different from one another. The most
frequent problem in migrating an application from one to another
is moving a set of programs which are designed around the
strengths and weaknesses of the system of origin without taking
into consideration the different characteristics of the target.
In many cases, such oversights are the result of time pressure.
In others, they are due to lack of information about which areas
need the most attention and how to attain the most improvement
for the least amount of work. This guide will help point the
way.
1-7
This is not a substitute for the various developers materials
available for RSX but is rather a summary intended to assist
third party software developers in assessing the desirability of
moving to RSX and to minimize the number of pitfalls into which
such a developer might fall. If you are new to RSX it will be of
assistance during those first days in which adjusting to any new
operating system is difficult.
The principles and techniques described are largely independent
of the implementation language to be used. An efficient RMS file
design will improve the performance of an application program
whether it is written in Fortran, Basic, or Pascal. Many of the
persons reading this guide will be doing so in preparation for a
migration of an existing application from CTS-300 or CTS-500 to
RSX and VMS, and some special notice is taken of this path.
1.2 Intended Audience
This guide is addressed to a number of different people with a
variety of backgrounds. You might be a third party software
developer who is considering a move to RSX. You might have
already decided to give RSX a try and are wondering how to plan
the migration of your software. You could be part way through a
conversion and are finding difficulties in locating the
information you need. You might be facing a particular problem
and are looking for a quick solution.
It is assumed that you know something of Commercial Application
development and have a favorite programming language. If you are
coming from a CTS-300 environment you may find that many of the
facilities and concepts available with RSX are complex. If you
are coming from VMS on the other hand, RSX will seem more
familiar.
1.3 On the Synergy of Operating Systems and Applications
Computer operating systems are alike in that in one way or
another they serve as a mechanism by which system resources are
allocated to various objects, terminals, or application programs.
They are different, however, in the overall design point which in
turn determines the type of application that is expected to be
layered on top. Some are designed to react as quickly as
possible to an event, some are designed for security against
power or component failures, some are optimized for a particular
language.
In all cases there is a form of synergy between an application
and the underlying operating system. Since each system has a
distinct set of strengths and weaknesses it behooves the
application developer to make such changes as will take
advantages of the strengths and avoid the weaknesses. Problems
1-8
do not usually reside exclusively in the operating system or the
application and most often result from attempts to treat one O/S
as though it had the properties of another.
The importance of an application designer making the investment
to understand the characteristics of a given operating system
cannot be overemphasized. The return on such an investment can
be staggeringly large. In one case a simple change of subroutine
linkages (which took 20 minutes) resulted in a 2X performance
improvement. In another case an optimization of a single
subroutine resulted in a 10X improvement of a critical module.
1.4. Elements of Application Tuning
There are two, distinct aspects of design and tuning of a
particular application in a commercial environment. The first is
the number of resources which are required by a particular
program such as the number of files, CPU cycles or terminal
operations. The second is speed and efficiency with which the
operating system can deliver those resources.
Identifying the contributions that an application makes to system
load is difficult for many developers. A module may work
perfectly well with small numbers of users but when a full
production load is attempted the overall system deteriorates
quickly.
All too often this condition will not occur until the product is
actually installed at a customer site where there is less time
and fewer opportunities to investigate the problem. The purpose
of this document is to help you anticipate such problems and to
apply corrective action as early and inexpensively as possible.
When a performance problem occurs, the solution must be to reduce
the number of requests for resources or to increase the speed
with which the requests are satisfied. Since the delivery of a
resource can never be made instantaneous, the first effort should
be to eliminate the requirement. The sections which follow
describe ways that you may reduce or eliminate the demands made
upon an operating system by your application.
1-9
Chapter 2
Overview of RSX
Announcement of the PDP-11 resulted in an industry demand for a
high quality, high performance and comprehensive multi-tasking
operating system. The RSX family began with an initial
adaptation from the PDP-15 and grew with additions of RSX-11D,
-11A, -11B, -11S, -11M, IAS, P/OS, -11M-PLUS and, preserving the
family ties with the 32 bit big brother, VAX-11 RSX. Many of the
concepts of VMS can be traced to previous work done on RSX.
2.1 Historical Overview.
A number of operating systems were developed for the PDP-11
including RT-11 as a 16 bit follow-on to OS/8, DOS-11 for larger
machines and more sophisticated customers, and RSTS for the
Education market. DEC mythology has it that at one time or
another there have been 27 different PDP-11 operating system
projects funded and underway.
But in the early years the scientific/industrial marketplace was
of primary importance to Digital and RSX was the flagship
product. Most of the early minicomputers were used for process
control and data collection and it was essential that the system
software offer the greatest possible degree of flexibility.
Hardware was expensive and if a month or two of bending and
squeezing the software meant that memory requirements could be
reduced by 4KW it was well worth the effort.
Furthermore, there was no end to the variety of configurations
which could be assembled. OEM's and end users were willing and
able to create their own hardware components which could have
characteristics ranging from a simple parallel digital interface
to DMA communication ports. And, since their world was primarily
real time, they required that response times to external events
be both extremely fast and highly predictable. RSX was designed
to satisfy the most exacting real-time requirements and could be
tailored to almost any configuration of off the shelf and custom
hardware.
Every feature has an associated cost. The extreme modularity of
RSX meant that the process of creating a working system was
extremely complex even if the hardware configuration was
comparatively simple. Users who didn't have critical real time
requirements were nevertheless obliged to deal with a human
interface designed for operating four or five processes at once.
This was particularly true for the less technically oriented
commercial users who were beginning to experiment with
minicomputers as an alternative to batch processing. Multi-user
protection was not supported in the early versions and the
2-10
so-called command language bore an unfortunate resemblance to
certain UNIX shells. Memory was divided up into partitions which
always seemed to be mismatched with the job at hand, the system
would simply stop if a particular resource were exhausted, there
was no data management facility other than block I/O and the only
implementation languages were Macro and Fortran. Terminals
seemed to be supported only as an afterthought.
But even scientists tire of creating their own record management
routines and over the years RSX was improved in ways which were
also attractive to non-technical users. As the underlying
hardware became less expensive it was possible to ease the
extreme requirements for partitions in memory and provide more
facilities in the executive. RMS finally appeared with multi-key
ISAM. Partitions and pool management became dynamic, round robin
scheduling and swapping support were added. DCL was provided
(albeit only as an "alternative" to MCR), DECnet was put in place
and, wonder of wonders, the terminal driver was made full duplex
and could even support type-ahead, trap CTRL/C's and process
escape sequences. The SYSGEN problem remained, and it was
finally decided to produce a pre-SYSGENed, bounded version of RSX
with commercial features which could be handled by a computer
naive customer.
The Micro PDP-11 was the first PDP-11 intended for installation
by a first time user. Since the hardware was user-installable it
was felt that the software should be as well. Moreover, it was
considered that the Micro system customer represented a new class
of prospective RSX user - a person not concerned at all with the
traditional scientific/real time features of RSX but simply
desiring a system which could quickly and easily be put to work.
Rather than require the user to describe the target hardware
configuration in detail, the software would accommodate itself to
whatever was available.
2.2 RSX Version 3.0
Micro/RSX was the first member of what is now the RSX Version 3.0
family and has been joined by RSX-11M-PLUS in both Pre-genned and
full kit form. Micro/RSX and Pre-genned M-PLUS eliminate the
SYSGEN process altogether and will adapt themselves to any legal
hardware configuration during system startup. M-PLUS still
requires a SYSGEN to move from the Baseline (distribution)
executive to full timesharing but even this process has been
considerably simplified.
The features available with RSX V3.0 include:
o Full support for all of today's PDP-11 high performance
CPU's, memory to 3.8 MB and I/D hardware where available.
o Support of all modern disks and peripherals.
2-11
o Disk structure, Named Directories and Logical Names
compatible with VMS.
o RMS including multi-key ISAM compatible with VMS.
o Terminal Emulation and Remote file access via DECnet.
o Shareable, Clustered and Supervisor-Mode libraries.
o Shareable, checkpointable Common Regions.
o Multiple Stream Print and Batch Queues.
o Implicit device spooling.
o Synchronous and Asynchronous I/O on up to 256 channels
per task (not supported by all languages).
o DCL with Extended Help and command line continuation.
o Disk Data Caching.
o Password encryption.
o Multi-volume backup to disks and tape.
Application implementation languages and tools include:
o FORTRAN-IV and FORTRAN-77
o DATATRIEVE-11
o DIBOL-83
o COBOL-81 and COBOL-11
o DBMS-11
o FMS-11
o Pascal
o RPG-II
o Sort/Merge
o MACRO-11.
With the availability of Version 3.0, RSX has finally become a
high performance, high functionality operating system base for
commercial applications.
2-12
Chapter 3
Getting Started
This section is for those users who have never installed or used
an RSX system. It is intended to get you on the air as quickly
as possible beginning with that uncertain moment your system has
been installed and checked-out and you're about to open the first
of the boxes containing the software. It will also outline the
general process by which the software is installed and customized
so that you may estimate the amount of time that will be
required.
3.1 The Three Faces of RSX
Beginning with the newly released Version 3.0, RSX comes in
three, highly compatible flavors (there are also two other
members of the RSX family, RSX-11M and -11S but these have become
special purpose products and are only of interest to the
traditional RSX customer audience).
3.1.1 Micro/RSX
The low end member of the V3.0 family is Micro/RSX which was the
first release of RSX with a full set of commercial features.
Micro/RSX was specifically tailored for the MicroPDP-11 and is
the first version which could be taken out of the box, installed
and used by a non-technical person. Micro/RSX served as the
basis for A-to-Z V1.0 and is usually the system of choice for
Micro-11 configurations.
3.1.2 Pregenned M-PLUS
The intermediate family member is the "Pre-Sysgenned" or RL02
version of M-PLUS. This is actually a variant of M-PLUS which
has been so configured that it will run on a wide variety of
hardware configurations including the MicroPDP-11. It is self
tailoring and, with the exception of certain features
specifically selected through SYSGEN, is identical to the full
M-PLUS system. This is the most appropriate system for
commercial users on configurations other than the Micro-11.
This version is only distributed on RL02 disk packs and there are
certain restrictions about which disk types may be used on the
target system. Otherwise, it is usable as soon as it can be
copied to the system disk. Unless you require special hardware
support or executive features which are not part of this version,
you should use this version of M-PLUS. Due to the size
constraints of the RL02 media, certain parts of the full M-PLUS
system are missing. If it is found that any of these components
3-13
are required on your system they may be copied from an M-PLUS
kit.
3.1.3 RSX-11M-PLUS
M-PLUS is the high end, full functionality system and must be
tailored (SYSGENed) before it can be used. It is distributed in
source form along with a Baseline System (minimal executive) with
which a SYSGEN may be performed.
The SYSGEN process is an interactive, question and answer process
by which the user describes the exact combination of executive
features and device support desired. The executive and
associated utilities are then assembled and linked together to
form a bootable system image. This image is further tailored by
setting up partitions, installing tasks, etc. until an executable
system executive is finished and ready to be installed. More on
SYSGEN later.
3.2 Installation and Startup
Once the three versions of RSX have been completely installed
there are only a few differences between them. The three
versions do, however, differ in the way they are shipped to the
customer and in how they are installed. These differences are
largely due to which assumptions can be made about the underlying
hardware.
3.2.1 Installing Micro/RSX
Installing Micro/RSX on a MicroPDP-11 requires only that the
software be copied onto the system disk. The distribution media
consists of either a set of diskettes or a single cartridge tape.
To be a legal MicroPDP-11 you must have either an RX-50 dual
diskette drive or a TK-50 cartridge magnetic tape. In either
case, the first unit of the media contains a bootable, stand
alone version of BACKUP. All that is necessary to install the
software is to insert the first media unit into the appropriate
drive and turn the machine on. Once the PDP-11 completes it's
self check it will notice that the media is in place and will
bootstrap BACKUP into memory.
BACKUP will announce itself and will then determine the processor
type. The distribution kit actually contains two complete
executives. One of the system images is intended for use on
PDP-11 processors that support separate instruction-space and
data-space mapping such as the 11/73 and 11/83. The other is for
use on processors which do not support I/D space mapping such as
the 11/23.
The disk drive which is to be used as the system disk will be
3-14
initialized (and cleared of all existing data!) and then the
system software will be copied into place. When the copy phase
of the operation is complete, you will be directed to remove the
media and restart the system.
The restart will request the time and date and will then run
automatically through the remainder of the startup procedure.
When the startup has completed the console terminal will log
itself out to prevent an unauthorized person from gaining access
to your system as the manager. The system is now fully
operational and ready for use. The entire process requires less
than an hour.
Micro/RSX was designed around systems with limited disk capacity
and consequently there are some optional software packages which
are distributed separately. These options are primarily those
required for software development. If you have these options
they may be installed right away or at some future time as
needed. The procedures for installing optional software on
Micro/RSX are described below.
3.2.2 Installing Pregenerated M-PLUS
The pregenerated M-PLUS kit is the quickest and easiest way
to get RSX-11M-PLUS running on your system and, for most
commercial customers, it is the most appropriate combination of
SYSGEN parameters. Installation involves copying the software
onto the target system disk, selecting the appropriate system
image file and installing the system disk handler. This process
is not as highly automated as that for Micro/RSX and the
appearance of the dialog is somewhat more mysterious, but the
procedure is only slightly more complicated.
The entire installation is described, step by step, in a special
chapter in the RSX-11M-PLUS System Generation and Installation
Guide. The process does not require any great degree of computer
experience as long as you have this guide handy when you begin
the installation.
The kit consists of a single RL02 which contains two, ready to
run system images along with all the necessary system files and
utilities. The RL02 pack can be bootstrapped and (except for the
lack of free space on the distribution volume) used straight
away. One of the system images is intended for use on PDP-11
processors that support separate instruction-space and data-space
mapping such as the 11/73 and 11/83. The other is for use on
processors which do not support I/D space mapping such as the
11/23. The non-mapped executive can also run on mapped systems
and it is, in fact, the system that runs when you first boot the
distribution disk.
The installation begins with bootstrapping the distribution disk
on the target system. When the startup procedure asks for the
3-15
time and date, the procedure is aborted and a special stand alone
Backup/Restore is bootstrapped from the RL02. This special task
will allow you to initialize the target disk, locating and
marking any bad blocks, and then copy all the contents of the
distribution kit onto the target disk. At this point all the
files are in place but the target disk is not yet bootable. What
remains is to choose the appropriate executive (mapped or
un-mapped), configure it for the target system hardware, and then
make the executive image bootstrappable.
These final procedures are largely automated although they will
require you to type in some very strange looking commands and
will produce even stranger looking output on the console. It is
from this type of dialog that RSX acquired it's evil reputation.
As of Version 3.0, however, it is no longer necessary to
understand the process in any detail. You will not be required
to make any decisions other than selecting an executive based on
your processor type. Simply follow the instructions carefully.
The last step begins with re-bootstrapping the kit disk. Once
again interrupt the startup procedure and, following the
instructions provided in the Sysgen and Installation guide, load
the driver for the system disk, bring it on line, and set default
to the desired system area. Selecting the proper system is the
only decision you have to make and, unless you have good reason
to do otherwise, will be determined by your hardware as described
above.
You then create a copy of the system image and invoke VMR which
performs an automatic procedure to tailor the selected executive
to your system disk. You then make the image bootable by
"bootstrapping" the just tailored image and then "saving" it in
such a way that the bootstrap code can locate the image file.
The installation is now complete. Remove the RL02 Distribution
pack and restart your system. The entire installation process
requires less than two hours.
3.2.3 Installing M-PLUS
Installing M-PLUS is a two stage process. First, you must copy
the information from the distribution kit to the target system
disk. Then, you must execute a SYSGEN to build an executive
which is tailored to your specific hardware configuration and
software needs. This process can either be performed on an
existing RSX system or on a new system.
Compared to previous versions, SYSGEN for RSX V3.0 is a painless
and largely automatic process. If your hardware configuration is
standard and you have no special requirements of the operating
system, you can be finished in a matter of hours. If you are
working on the target system you can direct SYSGEN to
'Autoconfigure' your hardware and it will use the results as the
3-16
default answers to the most difficult questions.
Furthermore, you can direct SYSGEN to create a 'Full
Functionality' executive thereby eliminating a large number of
intricate and confusing questions about executive options. If
you accept all the default answers, the dialog section of SYSGEN
is largely a matter of confirming that little used options are,
in fact, not desired.
If you are an inexperienced M-PLUS user, you are strongly advised
to use the Autoconfigure option and to accept all the defaults.
In this way you will get your system up and running as quickly as
possible. Once you have acquired more experience you may repeat
the SYSGEN to create an executive which is more closely tailored
to your needs. Most commercial users will find that the Full
Functionality executive suits their needs perfectly.
On the other hand, if you have an unusual hardware configuration
or if you have very special needs, you may wish to contract for
enough support to get you through the first installation. RSX
can be tailored to fit just about any hardware configuration
which can be manufactured, but the process of discovering all the
ins and outs of CSR and Vector assignments is a complex one.
This is no time to play the hero. If you have any doubts about
your own capabilities, call for help.
The full details of the installation and SYSGEN process are
beyond the scope of this section and are described in the RSX
System Installation and Generation Guide. There are a whole
series of different paths depending on what existing systems are
available but the simplest is installation on new hardware. The
sequence of events for installing on a new system is roughly as
follows:
Boot the distribution media. This will startup a special, stand
alone version of Backup (actually an RSX-11/S system image) which
you can use to format the target disk and scan it for bad blocks.
You then copy the first of two backup save sets from the
distribution media to the target disk. The result is a disk
which is hardware bootable and contains a Baseline executive
which will run on almost any hardware and which contains just
enough functionality to allow you to complete the SYSGEN.
Boot the target disk. The Baseline system will automatically
copy the second backup save set from the distribution kit to the
target disk.
Apply any necessary Updates. Unless you have obtained your
software shortly after a major release of RSX there will likely
be one or more Update kits which must be applied to the target
disk before you proceed with the SYSGEN. This process is largely
automatic. Invoke the UPDATE command procedure to apply the
corrections.
3-17
Invoke SYSGEN. Again, there are all manner of paths and options
but the best plan for a beginner is to go straight through from
the beginning to the end, accepting the defaults wherever
possible. While the simplest path does not require you to answer
many questions, a copy of the dialogue which takes place is
useful in the event that anything goes wrong. If you do not have
a hardcopy terminal attached as the system console it would be
wise to connect a printer to the CRT so that all the output will
be saved.
SYSGEN will ask a number of questions about how the executive is
to be built. Unless you know better you should take the default
in each case. The exception is the question labeled SU100 which
asks if you wish to run Autoconfigure. If you are running on the
target hardware you should answer YES. Doing so will result in
an Autoconfigure operation in which the Baseline system will
interrogate all the devices it can find on the bus and will
configure your peripherals accordingly. The only other questions
you will be asked will concern hardware which was expected but is
missing (why SYSGEN asks if you have a card reader is a mystery)
or about options that are costly or infrequently used (modem
support on terminal lines).
After about 15 minutes of dialogue SYSGEN will proceed to
assemble and link the executive and all the privileged tasks.
Watch the dialog as it unfolds and compare it to the example in
appendix D of the SYSGEN guide. The remainder of the SYSGEN will
take about three hours. The process is automatic but you will be
asked an occasional question so it's a good idea to check in from
time to time to see how things are progressing.
When SYSGEN completes you will have a fresh copy of the executive
but it will not be hardware bootable. Follow the directions in
the Guide to software boot the new executive and enter the time
of day. This action is only a sanity check to see that
everything worked properly. Then, SAVe the executive, again to
see if anything untoward occurs. The SAV will cause the new
executive to be rebooted and begin the standard system startup
procedure. If everything still looks all right, abort the
startup as directed and SAVe the system again, this time with the
/WB (Write Bootstrap) switch. This final step will change the
bootstrap blocks of the target disk to point to the newly
generated executive. The system will restart automatically and
you are on the air.
3.3 Logging In and other First Day Tasks
Once your system is installed and operational the first thing you
should do is log in as the system manager and make any additional
adjustments that you might feel are necessary. Typical chores
involve post-installation clean-up, selecting appropriate startup
options, creating user accounts, configuring terminals and
3-18
printers, and installing layered products.
3.3.1 Logging Into the Prebuilt Accounts
RSX is distributed with a System account and a User account. You
can log into either one from an unused terminal (such as the
console terminal once the system startup has completed) by
pressing the Return key until you get the system prompt (which
might be either ">" or "$"). Type "HELLO" and press Return. You
will then be prompted for a username and password. When these
have been entered and verified against the system accounting file
the login procedure will complete.
The prebuilt account username/passwords are SYSTEM/SYSTEM (or
MICRO/RSX on Micro/RSX) and USER/USER. Once the login task has
validated your account it extracts other information about your
account from the authorization file and then executes two command
files on your behalf. The first of these, LB:[1,2]SYLOGIN.CMD,
is executed for all users and, in it's most common form, will
determine your terminal type with SET TERMINAL/INQUIRE and then
chain to the LOGIN.CMD file in your default directory. If this
second command file exists it is also executed. In the case of
the USER account it will print out a welcome message and some
introductory information.
These two login files can be used to control the accounts in your
system. In the case of A-to-Z they are used to direct the user
directly from login into the appropriate A-to-Z menu.
RSX supports two Command Line Interpreters (CLI). MCR is the
traditional CLI still favored by the RSX old timers, and DCL
which will probably be more familiar to newcomers. The account
entry in the authorization file determines which of the two is
established for a particular user during login. You may switch
from MCR (usually identified by the ">" prompt) by typing
> SET /DCL=TI:
You may switch from DCL (usually identified by the "$" prompt) by
typing
$ SET TERMINAL/CLI=MCR
Most users learn one of the CLIs and stick with it. You should
modify the authorization file entry for each account according to
the user preference.
3.3.2 Post Installation Cleanup
The Pregenerated (RL02) version of M-PLUS contains both the
mapped and the un-mapped executives. Once the installation has
been completed you may wish to delete the unused files to recover
3-19
the disk space. A command procedure is provided for this
purpose. The System Generation and Installation guide details
how to proceed.
The procedure is completely automatic. It will determine which
executive is in use and will delete the unused (and completely
unnecessary) files. At its conclusion the procedure will report
the number of disk blocks recovered. You may postpone this task
if you have sufficient disk space for the early days of usage.
3.3.3 Customizing Startup
The Micro and RL02 systems are shipped to you with certain system
parameters, such as the number of batch and print queues, set to
values appropriate for an "average" configuration. These
parameters are stored in a file (LB:[1,2]SYSPARAM.DAT) which is
readable by non-technical users and which is interpreted by the
system startup procedure.
The statements within this file determine such things as the
amount of checkpoint space and secondary pool allocated, what
combination of batch and print queues are to be started, what to
do about error logging, and so on. Provision is made for
controlling every commonly used element of the system. If you
can use an editor, such as EDT, you will have little difficulty
modifying this file to your own taste. It is unlikely that you
will have to do anything further.
If you have chosen M-PLUS you will be supplied with a skeleton
system startup control file, LB:[1,2]STARTUP.CMD. This sample
file is sufficient to get you on the air but it is expected that
you will want to modify it and will know how to do so.
Regardless of the system, if you have added A-to-Z and are
running primarily A-to-Z applications you will find that A-to-Z
manages all of the usual system startup tasks. If you have
requirements beyond those managed by A-to-Z you will most
probably know how to take care of them.
3.3.4 Configuring Terminals and Printers
When RSX configures your system it can detect the presence of the
various device controllers on the system, but in the case of
serial and parallel ports it makes no attempt to determine the
type of device, if any, which is attached. You have several
options:
If the devices are all terminals, you can let SYLOGIN.CMD do the
work with it's SET TERMINAL/INQUIRE which will poll the terminal
to determine it's type whenever a user logs in.
If the devices are a mix of printers and terminals and you are
3-20
running Micro or Pregenerated flavors, you can edit the parameter
file LB:[1,2]SYSPARAM.DAT and, following the examples contained
within, designate each of the lines or ports with the appropriate
device type and line speed. The next time your system is
re-started the devices will be identified and brought on line.
If you don't want to be bothered, or if you are installing a
system for a customer who may not want to be bothered, you should
consider adding A-to-Z to the system. A-to-Z includes a process
which runs at system startup which will poll all the available
ports and determine the device type and speed of each line. This
information is stored in a table where it is available for
inspection or modification by the customer. The advantage of
this is that if a terminal is moved or the speed of a device is
inadvertently changed, the customer can deal with the problem
directly without any special training.
3.3.5 Creating User Accounts
The procedures for adding, changing, and removing accounts are
the same for all three systems. A utility (ACNT) may be invoked
by any privileged user to make any desired changes to the system
accounting file (LB:[1,1]RSX11.SYS). Take care that this file is
not inadvertently deleted. If the file is deleted you may have
some difficulty regaining control of your system.
Before you enter a new account you should have available certain
pieces of information which will be required by ACNT. You will
be asked to supply a username, typically the user's last name,
which must be unique on the system, a password which is a "hard
to guess" word of up to 39 characters that validates the identity
of the user to the system, a User Identification Code (UIC) which
classifies the user according to group and access privileges, a
default CLI (DCL is easiest for most new users to learn), and a
disk and directory to contain the user's files. The items are
entered one at a time and the default directory will be created
if it does not already exist.
The new user may log in immediately after you have entered the
information for the account. There are several other items which
you may enter including a session identifier, first name, and so
on. You may also modify an existing account if you wish to
change any parameter. Note that beginning with Version 3.0 of
RSX you may opt for password encryption on an account by account
basis. Such passwords may not be examined once they have been
entered but they may be specified anew if the original has been
lost.
If you have A-to-Z you may wish to manage your accounts with the
facilities provided for the A-to-Z Manager. Unless you have
special needs you should find that the facilities provided by
A-to-Z are adequate and are much easier and safer to use.
3-21
3.4 Installing Layered Products
Installation on any version of RSX is largely a matter of copying
files into the appropriate directories, possibly task building
certain modules against the symbol table for the executive on the
target system, and adding the necessary initialization commands
to the system startup procedures. On M-PLUS this is usually
accomplished by a command file, supplied with the layered
product, which will conduct a dialogue with the system manager to
determine which, if any, options are to be selected and then will
perform all the necessary tasks. Some products require that the
system manager manually edit the system startup file and provide
the appropriate commands.
Micro/RSX has a much more automatic installation architecture
which makes it possible for a person with little or no computer
expertise to manage the installation. A-to-Z has combined
elements of both of these architectures. If you are installing
A-to-Z layered products you may do so via an option on the A-to-Z
Manager's Menu. The procedure is the same regardless of which of
the versions of RSX lie underneath.
3.4.1 Installation on Micro/RSX
The Micro/RSX user does nothing more than choose the product to
be installed (in case there is more than one product module in
the kit) and to provide physical assistance with handling the
media. The architecture and the layered product developer do the
rest including making decisions about the underlying hardware,
product specific installation and startup procedures, and
integration with other components on the system.
To install a product the user logs into the system manager's
account (or the manager's account for A-to-Z) and invokes the
optional software installation procedure OPTION.CMD (or selects
the application installation option from the A-to-Z manager's
Auxiliary Functions Menu). The system software then requests
that the installer identify the appropriate device drive
according to the media type, insert the media, select the
application to be installed, and then insert more media as
required. While the particular product may require some
customization there is nothing further required to get it running
on the underlying system.
3.4.2 Installation on M-PLUS
Layered product installation on M-PLUS (both the Pregenerated and
full Sysgen flavors) assume a more sophisticated system manager
who has more particular requirements. In this case the layered
product developer supplies a kit which contains a collection of
all the components of the product and (with the nicer products) a
3-22
command procedure with which the manager picks and chooses from
the different combinations.
The developer is still required to be sensitive to those aspects
of the product which may change from system to system such as
re-linking privileged tasks against the target executive or
relocation of files and modules across disks. The developer is
also responsible for providing all the information regarding the
installation, usually in the form of an installation guide. The
installation guide and the release notes are the basis from which
the system manager decides between various options or, in some
cases, redesigns the installation procedure altogether to satisfy
specific system conditions.
There is no set pattern for installation on M-PLUS. Generally,
the Installation Guide provided with the product will direct the
manager to log in and create a temporary directory. The guide
will also describe the conditions that must be set up before
installation can proceed. Certain tasks may have to be installed
or optional software may have to be moved into place. The
manager will then mount the media, usually a magnetic tape or a
disk pack, and copy the installation control file (traditionally
INSTALL.CMD) into the temporary directory. The command file is
then invoked and, through a dialogue with the manager, determines
what actions to take. Once the dialog phase is completed, the
command file will perform the remainder of the installation:
copying and converting files, linking tasks, and setting up the
database. The installation guide will also usually provide the
manager with information about what modifications must be made to
the system startup procedure in order for the product to operate
properly.
Products for M-PLUS differ considerably in the degree to which
the installation is automated. Some come with a list of
instructions for the system manager including general directions
on what to add to the system startup command file. Others are
more automatic. The trend is towards automation as the available
hardware resources become more plentiful and squeezing the last
free byte out of a file becomes less important.
An example of such automation is the A-to-Z Base System which not
only controls it's own installation to a high degree but carries
with it elements of the Micro/RSX layered product architecture.
This makes it much easier for a computer naive system manager to
install and manage an application mix. It also makes it easier
on the product developers since the kitting procedure for A-to-Z
applications on Micro/RSX and M-PLUS is the same. Only the media
is different.
3-23
Chapter 4
Terminals
Terminal I/O is a prominent aspect of modern commercial
applications on timesharing systems. It is, in fact, one of the
primary reasons that the old batch processing systems have been
replaced. It is also, unfortunately, a major source of system
overhead.
Problems related to this aspect of application design are
particularly insidious because the cost of moving characters to
and from a terminal is so easy to overlook. Most programmers
understand that opening and closing files is costly. Far fewer
consider what is happening when a screen is being painted.
4.1 Cost of I/O
The cost of a particular I/O operation can be divided into two
general categories. The first is the number of instructions
which must be executed to guide the data between the program and
the device controller (overhead). The second is the time the
device requires to move the data between the controller and the
outside world (latency).
With disks, much of the time required to fulfill an I/O request
is the latency due to the mechanical movement of the head and
spindle. Once the disk head has been moved to the proper
cylinder, large quantities of data can be transferred very
quickly, usually without CPU intervention. More on disks and
files in the next section.
4.2 Specific Cost of Terminal I/O
Terminals are another problem altogether. There is a short
latency period while a character is shifted into or out of a UART
register but the real problem is that there are ever so many more
I/O operations involved in moving a block of characters to or
from the terminal. A disk can transfer thousands of characters
in a single operation whereas most terminals make the transfers
one byte at a time.
4.2.1 Interrupt Service
Terminals can cause almost 1000 CPU interrupts per second and
each interrupt requires the executive to suspend processing to
service the device. Even worse, terminals are unpredictable on
the input side. Operating system software can keep track of the
head position of a disk drive and can make a guess at how long a
given operation will take. This permits more advanced operating
4-24
systems, like RSX, to optimize the queue of waiting I/O requests.
But there is no way that the system can predict how long it will
take an operator to press a character key - the computer simply
has to wait for the interrupt.
4.2.2 Scheduler Overhead
RSX considers the completion of any I/O request to be a
"Significant Event". Such events signal to the system scheduler
that there may have been a change in various lists of tasks
waiting to execute - the I/O completion may have unblocked a task
of higher priority than the current task thereby necessitating a
context switch. With even a small number of commercial
applications painting screens the scheduler may be executing
hundreds of times per second.
Excessive context switching (thrashing) is a factor in almost
every system performance problem encountered thus far. The worst
possible circumstance is when the various interactive tasks are
competing for memory and are being checkpointed to disk in which
case each character moving to or from a program causes several,
far more costly, disk reads and writes to take place.
So the cost of moving a single character is the sum of the code
to actually move the characters to or from the device plus any
program overlays which may have occurred plus the interrupt
service routines plus the context switching. When many hundreds
of characters are being moved back and forth every second the
system overhead is considerable.
4.3 Minimizing Costs
There is little you can do to reduce the number of characters
which must be moved to and from the terminal. You can, however,
do a great deal to reduce the operating system overhead
associated with terminal intensive applications by reducing the
number of I/O requests necessary to move the requisite
characters.
4.3.1 Blocking Output Requests
Whether or not DMA hardware is available it is very important to
gather small groups of characters together before sending them to
the terminal. Throughput improvements of 10 to 1 have been
reported when scattered output requests have converted to calls
to a central routine which assembles the characters strings into
a single buffer. The amount of processing to move the characters
to the terminal is the same in either case but if the number of
I/O requests is reduced there will be a corresponding reduction
in scheduler overhead.
4-25
It is not always easy to determine just how an implementation
language such as BASIC or DIBOL performs terminal I/O but making
the effort to find out will be well repaid. For example, the
following three DIBOL code fragments will produce output that
looks exactly the same to the operator:
WORST,
DISPLAY (CHAN,'A')
DISPLAY (CHAN,'B')
DISPLAY (CHAN,'C')
This results in three I/O requests and some unnecessary
interpreter overhead.
BETTER,
DISPLAY (CHAN,'A','B','C')
This eliminates some of the interpreter overhead but there are
still three I/O operations (and consequently three calls to the
scheduler, etc.) involved.
BEST,
DISPLAY (CHAN,'ABC')
Best of all. It is admittedly unlikely that the output data will
be so easy to assemble. Most often it will be necessary to feed
the data fragments to a subroutine along with a flag which serves
as a 'Do It Now' or 'More Coming' signal. But it's worth it.
4.3.2 Blocking Terminal Input
It wasn't so long ago that many operating systems were designed
to only accept data from an operator in line mode - a series of
characters terminated with the Return key. No Fortran programmer
in his/her right mind would want to see the characters one at
time or, Heaven forbid, a five character escape sequence.
And the operating system people were glad to avoid the headaches
of having to worry about mapping all the ANSI sequences into
terminator codes. They preferred to tell any developer who
wanted to know whether an input string had been terminated with
Up-Arrow or Down-Arrow to figure it out themselves.
But terminals kept getting smarter and even people working in
laboratories type with their elbows now and again so eventually
the operating systems began to support commercial features,
beginning with single character input and some going so far as
providing elementary block mode facilities.
There is, however, a certain amount of holdover code in the
language processors which was put in place, in those good old
days, to give the underlying operating systems the appearance of
block mode input and in some cases, this code can get in the way.
4-26
Thus, it is once again important for the designer to understand
how a high level language translates the input statements into
RSX directives. It is the case that some languages will convert
what appears to be a field level request into a series of
requests for single characters. DIBOL, for example, converts a
READS into a series of single character input requests.
The latest versions of such languages may, however, provide a
means of control over such actions. This is important because
grouping input characters or data has the same benefit that
blocking output has. It often makes the job of the developer
more difficult, and sometimes impossible, if the functionality of
an application is going to be preserved. If you cannot avoid
processing the input stream a byte at a time, then so be it. But
if it is at all possible to change the character-at-a-time to
post entry validation then you will enjoy considerable
improvement in overall system efficiency.
4.3.3 Use DMA Terminal Hardware
The more expensive DMA terminal interface hardware will reduce
the number of interrupts the executive must service to move a
block of characters to a device. The terminal driver is able to
treat the output side of the device much as it would a disk. The
output operation is initiated and, except for the bus cycles
stolen by the interface to fetch the next character, the CPU is
free to perform other tasks.
However, there will be no great improvement with such an
interface if characters are fed to it individually or in small
groups. The greatest gain will be seen when output data is
gathered into a single block and sent to the device with a single
I/O request. It is possible to output an entire screenful of
information including escape sequences for formatting and setting
character attributes.
4-27
Chapter 5
Record Management Services (RMS)
In the early days of RSX there was little in the way of
facilities for file or record management. The File Control
Services (FCS) offered routines to open and close files and a
Macro program could issue QIO requests directly to the disk
driver. There was provision for rudimentary sequential access to
records and random access to disk blocks but otherwise
application developers were left pretty much to themselves. The
need for something more was apparent and eventually RMS-11 was
developed.
5.1 Introduction to RMS
The RSX Record Management Services (RMS) is a language
independent collection of file and record management routines
which may be linked into a user program and called upon to
perform a variety of file and record related functions. Since
the in-file format is determined by RMS and not by the host
language processor the records within the files are also language
independent thereby providing a ready mechanism for exchange of
data between applications.
RMS has undergone several revisions since it's inception,
primarily to take advantage of more advanced hardware and
operating system features. The first versions of RMS were
actually a collection of object modules that had to be linked
into, and carried about with, the task image. The more modern
flavors take full advantage of memory management hardware and CPU
operating modes for improved speed and reduced disk space
requirements and activity.
The current version of RMS may be linked as a shared library
which reduces disk requirements for task images and eliminates
the overhead of disk overlays for the RMS routines. Or the RMS
library may be clustered with the language OTS which increases
the amount of virtual address space available to the application.
Lastly, it may be linked as a Supervisor Mode Library (on
processors which support Supervisor Mode operation) which does
all of the above and reduces the time required for context
switching between libraries. For further details, see the
chapter on linking.
Full access to all of the RMS features is really only available
through Macro. Each higher level language has had to make
trade-offs in mapping the traditional semantics of the language
statements to the facilities provided by RMS. Some languages
allow the developer to specify the characteristics of a file
being created with great precision while others do not. Some
permit update of files opened for Input and some do not. Some
5-28
language statements may cause the run-time library to execute
hidden RMS operations. The DIBOL WRITE statement, for instance,
will store a record into both occupied and un-occupied cells,
executing either an RMS PUT or UPDATE operation, as required.
Utility programs are provided with RMS which provide for
creating, converting, and re-organizing RMS files. These
utilities can be used interactively or can be spawned from an
application to perform their various functions.
5.2 File Organizations
RMS provides support for three major and two minor file
organizations. The major types are Sequential, Relative and
Indexed. The minor types are Stream and Undefined and will not
be discussed further here. The types of access which are allowed
to a file depend on it's organization.
5.2.1 Sequential
Sequential files are the simplest format and, in some
circumstances, offer the highest performance. A sequential file
consists of a header and a number of data records. The file is
loaded by writing a series of data records. Later, the records
may be retrieved in the order in which they were written.
When you create a sequential file you may specify that the file
will contain fixed length or variable length records. Fixed
length record format offers the most efficient packing. Variable
length record format offers the greatest flexibility and is the
more common of the two formats.
Sequential files are most appropriate for serial writing or
reading of records. Very high performance may be attained with
languages which support the multiple buffering and deferred write
features of RMS. There is no provision for deleting, replacing
or updating records once the file has been created. Some
languages permit opening a sequential file to append data to
existing records. Sequential files with fixed length records
allow limited facilities for random access but record interlocks
are limited and such use is not encouraged.
5.2.2 Relative
Relative files consist of a file header, a prolog and data
records. The header and prolog contain information about the
file and the records.
Data records are stored in groups known as Buckets. The size of
the bucket is declared when the file is created and each bucket
is divided into as many record cells as will fit. Cells are all
5-29
the same size and can hold the largest record. Maximum record
size must be declared when the file is created. Data records
within the cells may be fixed or variable length but the unused
space in a cell containing a short record cannot be put to other
uses.
Once the file is created, the records may be loaded serially or
randomly. There is a flag byte in each record cell which
indicates whether a record is present, has never been stored or
has been stored and then deleted.
Records may be stored, retrieved, updated and deleted, randomly
or sequentially. RMS automatically maintains a current record
context which determines which record is selected for the next
sequential operation. If a record is written to a cell beyond
the end of the existing data, the file is automatically extended.
New buckets are created and initialized by RMS, as required.
Relative files represent a good compromise between convenience
and efficiency. As long as buckets are densely populated data is
stored with comparatively little overhead and access to any
record in the file requires only a single I/O operation (not
counting window turns). Interlocks are provided for files which
have been opened for write access by one or more programs.
Interlocks are maintained on a bucket basis.
5.2.3 Indexed
Indexed files offer the most flexibility to the programmer. You
may access records directly or sequentially by a primary key and
any of up to 254 secondary keys. The keys may be located
anywhere within the record, may be contiguous or segmented and
may be one of several data types. The index structure is dynamic
and will increase in size and number of levels to accommodate new
records as they are added to the file.
Access time for all records is uniform whether the file is loaded
in sequence by primary key, randomly across the file or in
clusters. The number of I/O operations required to reach any
record in the file is a minimum and is approximately equal to the
number of levels in the index.
In effect, RMS exchanges the CPU time required to manage the
(comparatively complex) index for the far more costly mechanical
movements of the disk head. For small files there is little
benefit to using this organization but for files which contain
tens of thousands of records the savings are substantial. There
is a substantial amount of overhead carried with indexed files,
both in terms of disk space and the code required to manage the
file.
Indexed files require the most attention to their design. There
are a large number of design parameters associated with indexed
5-30
files and unless values for these parameters are selected
carefully, the file performance may prove disappointing. Due to
the complexity of indexed files and the great variety of usage
patterns it is impossible to provide defaults which are
universally suitable and the burden of design is left to the
developer.
5.2.3.1 Structure
An Indexed file consists of a header, a prolog and two or more
index levels for each key. The header describes the location of
the file extents and the prolog describes the structure of the
records and the index levels.
The lowest level of the primary index contains the data records
themselves which are grouped together into buckets. Higher
levels of the primary index contain copies of data record keys
and pointers to the appropriate buckets at the next lower level.
The highest level of the index is called the Root and consists of
a single bucket.
Each alternate key defined for the file has it's own index
structure. The structure of an alternate key index is the same
as the primary key index except that, in place of user data, the
lowest level contains Secondary Index Data Records (SIDR).
SIDR's contain alternate key values and pointers to data records
in the primary index.
There is only one copy of a data record in a file but there may
be several copies of the key field. Data records are collated in
the buckets by primary key and the buckets are linked together so
that sequential access can take place without any references to
the index.
5.2.3.2 Population
A newly created file may be populated by writing records
sequentially or randomly throughout the file. The index is
dynamic and will expand both horizontally and vertically as
required. When the initial allocation is exhausted RMS will
expand the file automatically by allocating buckets as required.
As buckets fill up they will be split to make room for new
records.
Populating a file with an application program is not necessarily
the best way to proceed, however. There are certain features of
RMS which pertain to Indexed files, such as Bucket Fill size,
which are only available to Macro. In addition, secondary key
indexes will often be convoluted since even if the records are
added to the file in order of ascending values for primary key,
the values for the secondary key are unlikely to be in an optimal
order. Both these problems can be overcome with the Indexed File
5-31
Load utility.
5.2.3.3 Index Activity
When a new record is added to the file, the primary index is
scanned to find the proper bucket and the record is inserted
according to it's primary key value. If there is not sufficient
room to store the record, the bucket is split and half the
records are moved to a newly created extension bucket. The next
higher level index is updated to indicate the existence of the
new bucket and the record is inserted. If the index bucket is
filled, it too is split and if the bucket being split is the
Root, a new index level is formed.
If the file contains alternate keys, then each alternate key
index must be searched until the lowest level is reached, a SIDR
record added or updated, splitting the SIDR bucket if necessary,
and so on. The amount of I/O required to store a new record or
update an old one is a direct function of the number of keys.
5.2.3.4 Storage Overhead
The storage overhead in a bucket is a significant factor in
indexed files and, unlike relative files, may increase over the
life of a file. One of the features of RMS is that each data
record in a file has a unique address and if the record is
deleted the address is never re-used. Because of this, deleted
records leave behind a residuum which can be as little as two
bytes (fixed length records, no duplicates for the primary key)
or as much as the entire record (fixed length records, duplicates
allowed for primary key).
Similarly, if a record moves from it's original location in a
file when a bucket is split, a Record Reference Vector (RRV) is
left in the original location. If you expect to be splitting
buckets or deleting records frequently you must carefully
consider the record type and key locations and the implications
for bucket overhead.
5.2.3.5 Interlocks
RMS provides full interlock support on a bucket-wide basis. Each
program which opens a file declares the operations that it wishes
to perform and also the operations that it will allow other
programs. These declarations are often supplied by the language
runtime system depending on the particular statement used and the
default values may differ from one language to another. If the
access declarations of all programs accessing a file are
compatible, processing proceeds. Otherwise, the program opening
the file with the incompatible access code receives a protection
violation.
5-32
5.3 File Design
The design and tuning of RMS files is an area of study unto
itself. The number of possible usage patterns for files is large
and it is not possible for RMS to anticipate the manner in which
a particular dataset will be accessed. There is no obvious limit
to the amount of effort you may put into improving your files but
there is usually a point beyond which additional effort might be
better expended elsewhere. You must decide for yourself when you
have reached the point of diminishing returns.
This section is not intended to be a comprehensive tutorial on
file design but rather to point the way to those areas where
small changes may have major impact on the performance of an
application as a whole. See the RMS User Guide for further
details about file usage and design.
5.3.1 Which Files to Design
Although it may seem obvious, you should limit your design
efforts to those files which are most critical to the performance
of your application. If a particular file is used only on
occasion there is little benefit to spending a great deal of
effort on optimizing it's performance. Concentrate your efforts
on files which are large, which have many simultaneous users, or
which have high levels of insertion and deletion activity.
5.3.2 Selecting an Organization
In general, the amount of processing and overhead necessary to
manage a file increases with the flexibility of the available
access methods. Sequential and Relative file organizations are
very fast and carry little or no overhead but they do not provide
all the access paths of the Indexed organization. You will
almost always attain the highest performance by choosing the
simplest file organization which can be made to provide the
access features you require.
5.3.3 Common Design Factors
The design parameters of a file are set when the file is created.
The more complex the file organization, the more parameters there
are to consider. There are some parameters, however, which are
common to all file types.
5.3.3.1 Allocation
All files allow an initial allocation of disk blocks to be
5-33
specified when the file is created. In the case of Relative and
Indexed files this allocation is mandatory since storage space
must be provided for the file prolog.
Allocating space for a file when it is created has two benefits.
The allocation takes place all at one time and the disk blocks
are likely to be located close together. Also, the file can be
loaded without the interruption of extend operations.
Unused space which has been allocated to a file cannot be used
for anything else. Once the file has been fully populated you
may return any excess blocks to the operating system by
truncating the file. If your language does not support the RMS
truncate facility, you may spawn DCL with a SET FILE/TRUNCATE
command.
5.3.3.2 Extend Quantity
If a file is created with less than the amount of space which
will be required for storage of the data you may wish to specify
an extend quantity. This is the amount by which the size of the
file is increased when additional space is needed. The extend
operation is carried out automatically by RMS so your application
may not be aware that it is happening.
The default extend quantity for all files on a disk is set when
the volume is initialized and this value may not be appropriate
for a given file. If the extend quantity is too big you may be
wasting space. If it is too small the file will have to be
extended frequently which will mean frequent interruptions in
file processing, scattered file fragments and reduced performance
of subsequent file processing. A commonly used rule of thumb is
to specify an extend quantity derived from some unit of
processing such as a day's worth of records.
5.3.3.3 Contiguity
The best single action you can take to assure optimal file
performance is to make the file contiguous.
To make a file contiguous you must allocate all the required disk
blocks when the file is created which may result in some wasted
space until the file is filled. It may also require that you
re-organize the disk with Backup to gather the unused fragments
into a larger, contiguous space.
5.3.3.4 Carriage Control
If the file is ever to be printed or otherwise output to a record
I/O device (printer or terminal) you may wish to specify a
Carriage Control attribute for the file. Carriage Control
5-34
determines the way in which RMS will separate the records as they
are being output. Instead of carrying the formatting information
with each data record, RMS stores a single format setting in the
file header and this setting is interpreted by the system
utilities when the file is printed.
The default setting is Carriage_Return which means that as each
record is output it is preceded by a Line Feed and followed by a
Carriage Return. You may override this setting by specifying
None. If you specify None you must store the formatting
characters as data within your records, that is, you must
explicitly include Carriage Return or Form Feed characters within
your data records. Some languages have a special file qualifiers
which are used for this purpose.
5.3.4 Designing Sequential Files
Sequential file design is primarily a matter of allocation. Most
such files are created with variable record lengths and that's
the end of it. You are allowed to specify that records shall not
span block boundaries but in most cases this practice is wasteful
and not recommended.
There is a special case situation involving fixed length records
which you may wish to use. If you declare the records to be of
fixed, defined length when you create the file you may use a
limited form of random access to the file. You may store and
retrieve records either sequentially or randomly. Access to this
facility depends on which language you are using and there is no
record interlock protection.
5.3.5 Designing Relative Files
Designing Relative files is also comparatively simple. Like
sequential you choose an allocation and extend size. Records may
be of fixed or variable length but the record cells which are
allocated will be of uniform size and large enough to contain the
longest record. RMS keeps track of the byte count of variable
length records so the application never sees anything except real
data.
The one additional design factor is determining the bucket size.
A bucket is the logical structure within which records are
grouped and the size you choose can affect the performance of the
file. RMS reads and writes buckets in their entirety so that
when you read a record from a file the entire bucket will be
loaded into a buffer in your program space.
Bucket sizes are specified in blocks (512 bytes). RMS will fit
as many record cells in a bucket as possible. Record cells are
equal to the defined maximum record length plus one flag byte
plus two length bytes if the records are variable length.
5-35
Large buckets are good for access which is generally sequential
or clustered. If your program reads a record and then reads an
adjacent record which is in the same bucket, RMS will fetch the
record without re-reading the bucket thus saving an I/O
operation.
Small buckets give better performance if the access patterns are
random or if the file is shared between two or more programs.
Interlocks are maintained by RMS on a bucketwide basis so that
reading a record from a file with large buckets will tie up more
records than if smaller buckets are used.
RMS also keeps a flag byte for each record cell in the bucket.
This byte tells whether the cell has ever been used and whether
it is presently occupied. RMS can therefore distinguish between
records which have never been stored, have been stored, and have
been deleted.
5.3.6 Designing Indexed Files
Designing Indexed files can be a complex process. You must
define key fields, decide on optimal bucket sizes, partition the
file into areas and populate the file. Any or all of these
decisions may affect the performance of the file. Here, more
than anywhere else, you must consider each feature in light of
it's associated cost and use only those facilities which are
really necessary.
5.3.6.1 Specifying Key Fields
When you create an indexed file you must specify how many keys
you will use and a number of parameters for each key. You must
indicate how many segments each key has and where they are
located, whether the key may be duplicated, what type of data it
is, and an optional null value.
Key placement within the record is not critical for static files.
If, on the other hand, records are to be deleted, the primary key
should be located at or near the beginning of the record to
reduce the amount of residual overhead in the bucket.
The length of the key is also important. Index buckets contain
key values and bucket pointers. The shorter the key is the more
index entries will fit in a bucket and the shallower the index as
a whole will be. Shallow index structures mean fast access.
Keys may be string, integer or packed decimal fields. Whether or
not you allow duplicate values depends on the application but you
should be aware that allowing duplicates can slow down writing of
records and increases the amount of overhead due to deleted
records.
5-36
Null values can be specified for alternate key fields. If a
record is stored and an alternate key field contains only the
null character then no entry is made for that record in the
alternate key index. If many such records are stored the
overhead of the alternate key index processing can be kept to a
minimum. If at some later time the record is updated and a real
key value is supplied, the alternate key index will be updated
accordingly.
5.3.6.2 Sizing Buckets
The same general rules govern bucket sizes in Indexed files that
pertain to Relative files. If access is expected to be primarily
sequential or clustered around certain key values, make the
buckets large. If access is expected to be random or the file
will be heavily shared, make the buckets small. The memory
penalty for large buckets is greater for Indexed files since RMS
will allocate two, bucket-sized buffers when an Indexed file is
opened.
Furthermore, you should consider the relationship of bucket size
and the number of levels in the index. The time required to
process an index bucket (computation) is negligible compared to
the time required to move to the next index level (I/O).
Therefore, the time to access a record will be directly
proportional to the number of levels in the index.
Larger buckets mean a shallower index and faster operations. You
should set the bucket size to the smallest size (program size
considerations) that will result in an index with three or four
levels including the data. Very large files may require five
levels total.
You must anticipate the amount of activity overhead in the bucket
due to splits and deleted records. If you expect the file to be
static, that is, it will be loaded once and then read many times
you need not allocate space for the overhead. If, on the other
hand, you intend to insert and delete a lot of records you must
allocate additional space.
Finally, you may specify a Bucket Fill Size when you create the
file. This factor is the percentage of each bucket that will be
used if the Mass Load bit is turned on during the initial load of
the file. This is of most benefit when records will be added
uniformly across the file. Leaving the extra space in each
bucket will reduce (but not necessarily eliminate) the number of
bucket splits which occur and therefore the RRV overhead. There
will most probably be unused space in some buckets, however.
5.3.6.3 Areas
5-37
RMS Indexed files may be partitioned into Areas as part of the
initial allocation. An Area is a portion of the file which is
set aside for a specific purpose. Each index may have up to
three areas allocated to it. Each Area may have it's own
allocation, extension size and bucket size. The primary
advantage of this partitioning scheme is that logically related
buckets are grouped closely together in a file and head movement
while traversing an index tree or scanning sequentially through
data buckets is minimized.
In the simplest case a file could be divided into an Index and a
Data area. When bucket splits occur, the extension buckets are
allocated from the appropriate area. Without Areas, index
buckets will be scattered throughout the file in such a way that
moving from one index level to another may require moving the
disk head a considerable distance.
Areas also allow a certain amount of optimization. You may be
able to improve performance by specifying a smaller index bucket
if data records are very large and access is primarily random.
Large index buckets should be used if records are small or access
to the file is clustered.
5.4 Tuning RMS Files
File design is only one aspect of what makes an application mix
go fast or slow. But it's a highly visible aspect and is one
with which developers feel most familiar and comfortable.
It is often the case that poor file performance results from a
lack of understanding that every feature of a system has an
associated cost and, in some cases, the cost may not be obvious
or may be described in such a way that it's significance is not
properly marked.
This section will point out some of the more common areas in
which unneeded features can burden an application. See the
RMS-11 User Guide for more details.
5.4.1 Avoid File Opens
One of the most expensive things you can do with a file is to
open it. RSX supports lots and lots of files in any given
directory and developers who are moving over from CTS-300 may not
be making effective use of the directory structure.
Once you have a file open you should avoid closing it since the
close usually means another open at some later time. There is
more program space available on RSX than any other PDP-11. You
may be able to keep files open on RSX that memory constraints
caused you to close on other systems.
5-38
5.4.2 Use the Simplest Possible File Structure
If you use a multi-key index structure the file will necessarily
be larger and slower than if you use a single key. A single key
file is more complex than a relative file and access to a record
with a known key is slower. Sometimes developers discover that
they can live without ISAM after all and the performance
difference is considerable.
5.4.3 Beware of Overloading Directories
Directory searches are no different from scanning any other
sequential file and records at the end take the longest to find.
Keep your directories small and balanced for fast access.
Also, you should realize that when you create a new file the
system has to search through the entire directory to see if a
file of the same name already exists. Consider re-using an old
file in place of creating a new one.
5.4.4 Pre-Allocate the File
If the file is to be static and the amount of data to be stored
therein can be estimated then you may wish to allocate the entire
file when it is created. This will keep fragmentation to a
minimum and improve processing speed. Such allocations must be
judged in light of total disk capacity and their affect on free
space.
5.4.5 Make Critical Files Contiguous
The cost of locating the different parts of a file can represent
up to 30% of the disk activity. Make your file contiguous.
Failing that, make the extents of the file as large as possible.
If a file which is contiguous is extended then it is no longer
strictly contiguous, although some of the benefits of the initial
contiguity still pertain in that the initial allocation is a
single large extent. If the file is further extended over time
you may wish to periodically restore it to a contiguous state.
You can do this from your application by spawning a
COPY/CONTIGUOUS command.
5.4.6 Substitute Code for Mechanical Movement
Disk activity almost always involves some mechanical movement,
often requiring tens of milliseconds to complete. It is often
possible to replace disk operations with program code. If you
are suffering a performance problem, try to determine whether you
could simplify the file by doing a little more work in your
5-39
program.
5.4.7 Distribute the Load
Multiple disk spindles very often can provide more throughput
than a single drive for a given total storage capacity. The disk
head itself is subject to a number of demands in addition to data
transfer such as directory manipulation and program overlays and
consequently may become a significant bottleneck. Even though
timesharing will continue while the head is being moved all the
other disk I/O requests will be delayed and since commercial
systems depend heavily on I/O this is a serious problem.
RSX supports overlapped seeks if multiple disk drives are
available so that a number of disk requests can be processed
simultaneously.
5.4.8 Avoid Going Overboard
Multiple keys are costly if records are being stored or updated
in a file. The number of disk operations required to store a
record is proportional to the number of keys.
Very long keys consume extra storage and can increase the number
of levels in an index by reducing the number of key entries in an
index bucket.
Keep the keys near the beginning of the record. For certain
types of files, deleted records leave behind a fragment which
includes the record from the beginning through the end of the
primary key.
Load the file in order of ascending key value. Doing otherwise
will cause more frequent bucket splits and a more complex index.
Avoid duplicate keys, if possible. Duplicate primary keys
increase the overhead of deleted records. Duplicate alternate
keys can result in long chains of pointers in the SIDR records.
5-40
Chapter 6
RMS Utilities
Most programming languages on RSX lack support for one or more
RMS features. Moreover, there are a number of operations which
should be carried out on a regular basis for creating and
maintaining RMS files. Therefore, special utility programs have
been provided which may be invoked interactively or may be
spawned with a command file.
What follows is a brief description of three of these utilities.
For details about the operation and use of these programs see the
RMS-11 Utilities Manual.
6.1 RMSDES File Design Utility
RMSDES is an interactive utility for designing and creating RMS
files. You design a file by issuing commands to set, clear and
display file attribute values in a file design buffer workspace.
Attributes are arranged in functional groups which pertain to the
target system, file, record, key and areas for the file. You may
enter or change the attributes in order from beginning to end, by
section, or individually. You may enter the attribute values
directly, load them from a description file or copy them from an
existing data file. You may create a file directly or save the
workspace in a description file for use at another time.
When the attributes are arranged to your satisfaction you may
create an empty file. RMSDES will check to see that all required
values are present and will perform sanity checks on the values.
6.2 RMSIFL Indexed File Load Utility
RMSIFL reads records from any type of RMS file and loads them
into an empty Indexed file. RMSIFL is superior to other loading
methods in that it bypasses the standard RMS mechanism and
exploits the underlying file structure to produce an output file
quickly and with optimal packing of buckets. Use of RMSIFL is
the most common way of honoring the Area Fill values specified
for the indexed file when it is created.
RMSIFL operates in a number of phases. It will optionally sort
the input records by primary key value before loading them into
the indexed file. During the load it will extract any alternate
key values and store them in a temporary file. When the data
records are completely loaded, RMSIFL will sort the values for
each alternate key and then build the alternate key indexes. The
resulting file is optimized for the fastest possible access along
all index paths.
6-41
RMSIFL has some limits. The index structures are created without
the assistance of RMS which means that the output file must be
empty at the beginning of the load. RMSIFL can handle no more
than 20 keys per record and may be limited to bucket sizes of
five blocks or less depending on the format of the record and the
keys.
These restrictions aside, RMSIFL is absolutely the fastest way to
load a high performance indexed file.
6.3 RMSCNV File Conversion Utility
RMSCNV can read data records from any RMS file organization and
load them into any other, either locally or over a network.
RMSCNV uses the standard RMS facilities to read the data and
build the output file. It is not subject to the same limitations
as RMSIFL.
RMSCNV can be used to append records from one file to another,
re-establish contiguity and convert data from one file format to
another. If the output file is to be sequential, RMSCNV can
create it. Otherwise the file must be created before RMSCNV can
be used.
A number of switches are available (and may be required) which
determine such operations as record truncation and padding,
recognition of bucket fill size and mass insertion mode, block
mode operation and so on.
When used with Indexed file structures RMSCNV is not subject to
any of the restrictions of RMSIFL but does not provide the same
level of performance.
6-42
Chapter 7
RMS for CTS-300 Developers
Since many commercial users are migrating their applications from
CTS-300, this section is set aside to point out some of the more
commonly encountered problems caused by the differences between
DMS and RMS files.
7.1 Record Locks
The biggest single difference encountered by developers moving
from CTS-300 to RSX is the way record interlocks are managed.
CTS-300 (and VMS) were created with the record management
facility tightly integrated with the operating system and it is
possible, therefore, to control access to shared files in such a
fashion that potentially dangerous operations can be avoided.
The most common such case is allowing one program, which has
opened a file for input, to read a record or bucket which has
been locked by a second program which has the file open for
update. This is permitted on both CTS-300 and VMS such that read
operations issued to a file opened for Input will never encounter
a record lock. There may, in fact, be delays in the read
operation if the request occurs at a difficult time but these
delays are temporary and are invisible to the application.
RMS on RSX is more of a layered application than an integral part
of the operating system. It is possible, therefore, that a
program which has a file open for input might try to traverse an
index tree which is being updated by another program and if this
happened, the program issuing the read might crash. To prevent
such an occurrence, any access to a file which has been opened
for update or write operations is subject to record locks.
7.2 Access Modes
As explained earlier, DIBOL makes certain assumptions in mapping
the various language statements to RMS facilities. One commonly
encountered example has to do with access modes declared when a
file is opened. When a program calls upon RMS to open a file it
must declare both the operations it intends to perform (Read,
Write, Update, Delete) and also the operations it will allow
other programs to perform. RMS matches these access declarations
with those of any other program which has the file open and will
only permit the latest program to proceed if the access codes are
compatible.
This has certain side effects. A common example is the complaint
that DIBOL on RSX is much slower than DIBOL on RT-11 or RSTS
since it takes longer to read sequentially through a file. For
some reason that is not clearly understood the sample file chosen
7-43
for such experiments is always a relative file which results in
an unexpected condition.
When DIBOL opens an existing file for input it tells RMS that it
wishes Read only access and, because of the traditions of
CTS-300, it will allow other programs write access. RMS
interprets this as a hint that the data in the file may change
(even though no other programs have the file open at that moment)
and consequently will re-read the entire bucket each time the
DIBOL programs tries for the next record.
If the DIBOL program were to open the file in Update mode, the
bucket would be locked when the first read occurs, and subsequent
reads take place directly from the bucket in memory. There would
only be I/O when moving from one bucket to the next. The
difference in processing speeds between input and update mode
will be directly proportional to the number of records in each
bucket.
That the open mode should affect the speed with which a program
may scan a file is just one example of how "poor performance" may
be due to a simple misunderstanding.
7.3 Interchanging Sequential and Relative Files
It is a common practice on CTS-300 to create a file as
sequential, fill it with records and then close the file and
re-open it for random access. This is also possible with RMS
except that the file must be created with a Relative structure.
While RMS permits limited random access to a sequential file with
fixed length records, the use of Relative files offers more
capability and is the preferred method.
There is one consideration in transporting such an application
from CTS-300 to RSX or VMS. It is sometimes the case that such
files begin with a header record which is a different length than
the transaction records. Since RMS relative files are allocated
in record cells, each large enough to contain the largest
possible record, it would be a wasteful to write a very large
record followed by a number of smaller ones. It is much more
efficient to write the header data into a number of smaller
records and adjust the transaction record numbers accordingly.
7.4 Extending Files
RSX permits files to be extended regardless of the internal
structure while CTS-300 does not. This means that you need not
allocate all the space required for a file initially.
Performance will suffer somewhat if the file is extended many
times but you can overcome this by periodically using RMSCNV to
restore the contiguity.
7-44
7.5 Print Files
It is common practice on CTS-300 to write records in fragments
using mixtures of WRITES and DISPLAY statements. This is done
partly for coding convenience and partly due to the increased
flexibility of formatting. This practice is possible because DMS
files are really an unstructured stream of characters whereas RMS
files (on VMS as well as RSX) are a series of records.
Formatting of reports on RMS is provided through the Print Mode
files (O:P) which use the NONE declaration for Carriage Control
when the file is created. Each subsequent output operation,
whether a WRITES, DISPLAY or FORMS creates an individual record
and the application program is responsible for adding all the
carriage control characters. While these files are somewhat
lacking in storage efficiency, the programmer has full control
over the output format.
A consideration when using such files is that attempts to re-read
the file must be made with care. On CTS-300 it is possible to
create a record in a file with a number of DISPLAY statements
followed by a WRITES. On re-reading the file, all of the
information will be returned as though it had been written as a
single record. With RMS this will not be the case. Each field
will be stored as an individual record and will therefore be
returned as individual records in exactly the order with which
they were written.
7-45
Chapter 8
Flow Control
Most small system commercial applications are broken down into a
(possibly very large) number of program modules, each of which
will fit within the bounds of a limited hardware configuration
and CPU architecture. Flow control is that aspect of application
design which determines the manner in which these logic modules
exchange information and control.
One of the dimensions along which operating systems differ most
widely is the variety and comparative efficiency of the flow
control mechanisms. CTS-300 (RT-11) and RSTS are limited to
program Chaining. VMS allows a primary program module to Chain
to a secondary, spawn a secondary in a sub-process or, when the
secondary module permits, call it as though it were a subroutine.
RSX permits both chaining and spawning - each in a variety of
flavors. Each of these mechanisms has it's advantages and
disadvantages.
RSX had it's origins in the Scientific/Industrial marketplace
where Real Time response is a top priority. The design point was
to create a system in which a particular program module (Task)
could be activated as quickly as possible upon the occurrence of
some event. Consequently, the flow control architecture is more
complex than the other systems, and it requires a bit of
understanding before it can be used to greatest advantage.
8.1 Concept of a Task
The fundamental unit of execution on RSX is the TASK. Compilers
and editors are tasks. Application programs are tasks. Even
device drivers are tasks. Activation of a task is a two step
process.
8.1.1 Installing a Task
Before RSX can do anything with a task it must be Installed.
Installation is the process by which the location and
characteristics of the executable module are made known to the
system executive. Once installed, a task is available for
execution and remains so until it is removed.
Each task on the system must be installed with a unique, six
character name. During it's installation the executive records
the task's location on disk (by File ID for very fast access),
it's priority, the name of the partition in which the task will
execute, and several other items which reduce to an absolute
minimum the number of operations which remain before the task can
actually begin to execute. Contrast this with other systems in
8-46
which a RUN command will cause the executive to begin looking
through disk directories just trying to find the program image.
8.1.2 Task Names
Installed tasks are identified by name and therefore the name
must be unique on the system. Multiple programs may have the
same task name but only one of them may be installed at any time.
The name of the task may be established when it is linked, when
it is installed, or when it is requested and run.
A special case of task naming is the 'Prototype' name which
consists of three periods followed by three alphabetic characters
such as "...PIP". This is the mechanism by which multiple
terminals can be executing independent copies of the same
program. When a particular terminal invokes a task which was
installed with a prototype name, the executive creates a private
copy of the task for the requesting terminal and gives it a
unique task name by combining the prototype name and the terminal
name. Thus, a copy of PIP running at terminal 11 becomes
"PIPT11". The prototype task name is an important facility for
systems where multiple copies of job streams may be executing
simultaneously.
8.2 Requesting an Installed Task
Once a task is installed and brought to the absolute brink of
execution, all that remains is for some entity on the system to
request it's execution. Some tasks are requested as a result of
a command typed at a keyboard. Others are requested via system
directives issued by another program. There are a number of
different mechanisms involved but the functionality of all of
them falls into one of three categories.
8.2.1 RUN a Task
The simplest way to invoke a task is to RUN it. The RUN command
has four forms.
8.2.1.1 Running an Installed Task
If a task is already installed you may invoke it with the RUN
command and the task name:
RUN taskname
8.2.1.2 Running a Task from a File
If a task is not installed you may invoke it directly from the
8-47
task image file:
RUN filespec
This form of the RUN command invokes a special facility known as
Install-Run-Remove. Before the task actually runs it is
installed with your terminal name (e.g. "TT11") as the taskname.
The task is then run and when it exits, it is removed. You may
pass a command to the task with the /COMMAND:"command string"
qualifier.
8.2.1.3 Running a System Utility
You may invoke the system utilities by preceding the task image
filespec with a dollar sign:
RUN $RMSDES
This tells RSX to look in the system library for the image file.
You may pass a command to the utility with the /COMMAND:"command
string" qualifier.
8.2.1.4 RUN as a Scheduling Command
The last form of RUN allows a privileged user to schedule the
execution of an installed task for some future time:
RUN/SCHEDULE:hh:mm:ss taskname
8.2.2 Chaining between Tasks
The chain mechanism on RSX is the RPOI$ directive - Request and
Pass Offspring Information. All the popular implementation
languages provide support for this directive in one form or
another. Check the User Guide for details.
There are two options associated with the RPOI directive of which
you should be aware. One option determines whether the program
issuing the directive exits. The other determines whether RSX
Parent-Offspring connections are passed to the target task.
Unless there is a good reason to do otherwise, both these bits
should be set.
The directive itself may be used in two ways according to whether
the offspring task has been installed.
8.2.2.1 Indirect Chaining (to a non-installed task)
You may chain to a non-installed task image file indirectly by
8-48
forming the task image filespec into a pseudo RUN command and
then RPOI to either MCR or DCL passing the string as a command.
This is how DIBOL and BASIC implement the STOP 'filespec' and
CHAIN 'filespec' statements. This form will be the most familiar
to developers who are used to working with CTS-300 and RSTS but
it is also the least efficient. This is really a special case of
the RUN filespec command and because of the number of
intermediate tasks which must become involved it is the slowest
form of chain.
8.2.2.2 Direct Chaining (to an installed task)
You may chain directly to an installed task by using the
installed task name as the object of the RPOI$ directive. DIBOL
supports this form with the CHAINI subroutine. You may pass a
command string to the offspring and the performance is much, much
better than the indirect chain.
8.2.2.3 Chaining to a Command File
You may chain to a command file by issuing an RPOI$ to the
Indirect Command File processor and passing the the name of the
command file to be executed. The last action of the command file
should be to invoke the next section of your application.
8.2.3 Spawning an Offspring Task
Developers who are moving from CTS-300 or RSTS may, at first,
have difficulty understanding the significance of the Spawn
facility but it is one of the principal reasons that RSX was
chosen as the first operating system for A-to-Z.
It is the foundation of the A-to-Z application integration
architecture in which commonality of form and function is
attained by spawning the appropriate module to perform an
operation such as spawning Business Graphics to display some
data.
It also serves as the mechanism by which otherwise unrelated
applications may be nested such that the operator may request an
ongoing task to be suspended while another function is performed.
That function may also be interrupted and so on.
With or without A-to-Z, Spawn allows an application to invoke
almost any facility on the system without losing any context
other than, possibly, the contents of the screen. You can spawn
a system utility with a command to create an ISAM file, for
example, and wait for the exit status to determine the success or
failure of the operation. You can spawn infrequently used
sections of your own code as independent tasks to avoid having to
build them into the mainline images. You can spawn the CLI tasks
8-49
to execute commands as though they were being entered from the
keyboard. The possibilities go on and on.
Like Chain, Spawn has two forms depending on whether the
offspring task is installed or not.
8.2.3.1 Indirect Spawn (of a non-installed task)
You can spawn a non-installed task indirectly by spawning your
favorite command line interpreter with a 'RUN Taskname' command.
You can include the /COMMAND:"command string" qualifier for the
RUN command to pass a command string through to the task.
You should also use the /STATUS:TASK qualifier as this will
assure that the exit status which is passed back to the parent
task will be that of the task and not the RUN command itself.
Because it carries all the overhead of the RUN command entered
from the keyboard, this is the slowest of the two forms of spawn.
8.2.3.2 Direct Spawn (of an installed task)
Direct Spawn of an installed task is the fastest way to invoke an
external facility on RSX. You may pass the task a command string
and you may wait for exit status. You can spawn more than one
offspring task at a time if your implementation language
interface to spawn permits.
The designer of the parent task is responsible for deciding
whether to wait for the offspring task to exit and for correctly
interpreting any status which is returned. If task A spawns task
B which in turns chains to task C passing all connections and
task C exits, the status returned to A is that of C, not B.
8.3 Parent-Offspring
You may choose to create a library of spawnable tasks for your
application. These tasks may perform infrequently used functions
or they may serve to centralize certain functions as in the case
of multiple front end tasks feeding transactions to a single (or
multiple) background processor. The two most commonly used means
of communication with the parent task are the command line buffer
passed from the parent task to the offspring and the status code
passed from the offspring back to the parent task.
8.3.1 Command Line
RSX provides for passing a 255 byte command line to any offspring
task. The offspring must retrieve this buffer through use of the
GMCR$ directive. Access to this directive is available in the
more popular implementation languages. This buffer may contain
function codes, filenames or any other data in a format agreed
8-50
upon by the parent and offspring task designers.
8.3.2 Status Code
RSX provides for a task which is Exiting to pass back a 16-bit
status code to the parent task. The directive is EXST$ and
access to this directive is available in the more popular
implementation languages.
The low order three bits of this code are defined (by convention)
to be the error code and severity level. The remaining bits may
be used to communicate any other information in a format agreed
upon by the designers of the parent and offspring tasks.
A-to-Z has established a convention by which this 16 bit value is
divided into fields, each with a particular meaning. You might
consider adopting this convention.
8.4 Intertask Communication
RSX provides two principal mechanisms for communicating between
tasks running on the system.
8.4.1 Send/Receive
Send and Receive can be used to pass blocks of information
between cooperating tasks. The block can consist of up to 500
bytes of information. If more data is to be passed it can be
done with multiple blocks or by storing the information in a file
and passing the name of the file in a message.
A second, more complex mechanism allows passing of a memory
common between two tasks. This feature is more complex than most
applications require and the memory mapping requirements make
it's use in commercial applications uncommon.
Support for Send Data and Send By Reference varies from one
language to another. Some languages provide support through
library subroutines while others allow access to the requisite
system services.
8.4.2 Global Event Flags
RSX supports a second intertask communication mechanism which is
specifically intended for synchronization of task execution. It
is possible for a program which has access to system services to
define and manipulate event flags on a task, group or system wide
basis.
Event flags are one bit registers which can be set, cleared or
8-51
interrogated via system services. Typically one application will
set a flag whenever data is available for processing by another.
The waiting application will clear the flag when all available
data has been processed.
Support for event flags is limited or non-existent in some
languages. It is mentioned here because it is the highest
performance intertask signaling mechanism available on RSX.
8-52
Chapter 9
Linking and Overlays
Probably the single biggest surprise to developers who are moving
to RSX from other environments is the linker - The Infamous Task
Builder (TKB). It's very, very slow. We know it's slow, we use
it too. There's no getting around it. No matter which
implementation language you have chosen you will sooner or later
have to deal with the Task Builder.
It's also an exceptionally powerful linkage editor. With it you
can form your program into a monolithic or overlaid structure,
hook in shared common regions, set execution priority, pre-open
I/O channels, make use of Supervisor Mode shareable libraries,
patch global references, set stack sizes, even get cross
reference tables.
If your language allows, you can create Multi-User (shareable
task images). You can even arrange that disk overlay segments be
loaded asynchronously. The cost of all this functionality is
that the Task Builder itself executes very slowly compared to
RT-11 or VMS.
Again, this section is not intended to make you an expert but
rather to get you started as quickly and as easily as possible.
See your language RSX User Guide for further details.
9.1 Invoking TKB
TKB commands and parameters can be entered interactively but, for
any but the simplest programs, you will find it much easier to
use it with command and map files.
The Command file (.CMD) contains the input and output file
specifications and all the option statements.
The Map file (.ODL) describes the location of the object files
and the manner in which they are to be arranged and/or overlaid.
If an ODL file is to be used the name of the file is supplied in
the CMD file in place of the list of input modules.
Most of the high level languages (DIBOL, BASIC, COBOL) have a
'build' option of some type which will construct prototype CMD
and ODL files for you. The CMD files can often be used as is.
Unfortunately, it is not possible for the various compilers to
anticipate the way your subroutines will call each other so it is
left to you to design and describe the module overlay structure,
and edit the ODL file accordingly.
9.2 Disk Overlays
9-53
As an application designer you may choose from a number of
options with regard to overlays. You may use a single overlay
tree structure or you may use multiple, co-trees. You may
specify that the overlay segments be loaded manually or
automatically. You may direct that the overlays be loaded from
disk or be mapped from memory.
Overlaying subroutines from disk on RSX is done as on most other
systems - a call to an entry point in the routine is actually
diverted to a system (linker) supplied autoload vector which
checks first to see if the segment is already loaded and if not,
issues a read request to the program image on disk to load the
blocks which contain the desired routine.
Once the routine has been loaded into the proper address range,
the call is allowed to complete. Note that this mechanism is
only intended to work for a call. Attempting to reference data
in overlay segments will generally not work unless you load the
segment manually.
Overlay segments may be specified as manual load or autoload.
Only the latter mechanism is of interest to most commercial
application developers.
9.2.1 Trees
TKB requires that overlays be strictly tree structured, that is,
the Main Program module forms the root or trunk of the tree and
the subroutines are grouped into segments which form the
branches. When a module which is lower in the tree calls a
module which is higher up, the intervening segments are also
loaded into memory.
Subroutines may not call each other across branches and TKB
assures that global symbols are resolved uniquely in each path of
the tree. Thus, if routines in two or more branches of the tree
call a common routine, and that routine is not linked into the
root (routines in the root are always available to be called from
everywhere), then there may be two or more copies of the routine
in different places in the task image on disk.
If such a single tree structure is used, TKB will assure that the
references throughout the tree are correct and proper. It is, in
fact, the searching up and down the branches of the tree to
assure that there are no ambiguities in the intercall structure
and that the return paths will always be protected, that consumes
much of the execution time of the task builder.
9.2.2 Co-Trees
Developers who are used to other sorts of overlay structures,
9-54
such as the Regions used on RT-11 may find that task images
become monstrously large due to the requirements for unique
pathways and hence duplication of code many times throughout the
structure. It may, in fact, be impossible to attain certain
kinds of functionality, particularly if subroutine context is
expected to be maintained across calls to the routine.
In such a case you may use Co-Trees to emulate the RT-11 region
structure. To do this, you define multiple root segments, one
for the root segment and one for each 'region'. The modules
which are to reside in each region will then be linked to the
appropriate root. These extra root segments may contain
application code or they may be dummy (null) segments created
with NAME directives in the ODL file.
Co-Trees allow a much greater degree of flexibility in terms of
calling segments from different parts of the program and
following different paths to a common routine. However, it is
impossible for the task builder to determine what order calls
will occur in and there is no protection against calls which
might cause the return path to be destroyed.
The following (oversimplified) command sequence will direct the
RT-11 linker to create a program with a root and two overlay
regions. MAIN can call any of the four subroutines directly and
the routines can call each other as long as the return path is
preserved.
PROG=MAIN/C
SUBRA/O:1/C
SUBRB/O:1/C
SUBRC/O:2/C
SUBRD/O:2
This (oversimplified) ODL structure will direct the task builder
to create an overlay structure which is functionally identical to
the one shown above.
.ROOT MAIN-REG1-REG2
.NAME NULL1
REG1: .FCTR NULL1-*(SUBRA,SUBRB)
.NAME NULL2
REG2: .FCTR NULL2-*(SUBRC,SUBRD)
The RT-11 overlay regions and RSX co-trees are so similar that
translation from one to the other is almost entirely mechanical.
9.3 Memory Resident Overlays
Memory Resident overlays work by changing the virtual address
mapping registers for the task. When the program is initially
loaded into memory the entire task image is loaded from disk.
9-55
When a call to an overlay segment takes place, the call is once
again diverted to the autoload vectors, only in this case the
segment load is accomplished by changing the memory management
mapping control registers so that the subroutine 'appears' in the
correct place. Since there is no wait for the disk transfer to
complete, the overlays are much faster.
There are, however, a number of restrictions. Because of the way
the memory management hardware works the memory resident overlay
segments must begin on 8 KB memory boundaries and the segment
size may not exceed 8 KB. Also, the task builder locates the
overlay segments in such a way that the task 'upper limit' cannot
be extended in memory. The former restriction is a nuisance but
the latter is more severe, particularly for language processors
which allocate 'heap' storage dynamically.
Of the popular languages on PDP-11's, only Fortran allows the use
of memory resident overlays. The use of Disk Data caching in RSX
V3.0 will, however, improve the speed of disk overlays to almost
that of memory resident.
9.4 Guidelines for Use
Overlays do not come free. Even the memory resident overlays
require a fair number of machine instructions to be executed.
You will find, therefore, that if overlays are misused there may
be a problem with performance. In particular:
Disk overlays are just like any other kind of disk I/O. There is
a delay while the autoload code waits for the QIO to complete and
there is an opportunity cost when the disk head is moved away
from the data files.
You should carefully consider the flow of control through your
subroutine structures. The best possible design is where the
flow is sequential through a series of segments in the same
region with each performing it's particular function in turn.
Many application root segments are nothing more than a common
data region for context and a series of calls which invoke a
stream of modules moving through the segment.
The Worst possible design is one in which two, co-resident
routines are called repetitively. There was a case in which the
keyboard input and output routines of a program occupied the same
region but were in different segments. Every time the operator
pressed a key there were at least two disk read operations. The
performance of the application doubled when these two routines
were moved to the root.
9.5 Linking to RMS
RMS is provided in the form of one or more subroutine libraries
9-56
which are, insofar as is possible, language independent. All of
the high level language compilers translate the various I/O
statements into a call to one of the RMS routines and provide
such mapping of arguments as may be required for proper
communication.
The first version of RMS was nothing more than a collection of
object modules which had to be linked directly to the application
program. As hardware and software technologies improved it
became possible to shortcut many of the linkage processes while
at the same time improving task sizes, linking time and
processing speed.
RMS is provided in several different forms and each has it's own
linkage method. You must select the flavor and the linkage when
you task build your program.
You may chose between a variety of linkages, usually without
changing any aspect of your application code.
9.5.1 Disk Resident RMS
It is still possible to link the RMS modules directly to your
program. The code is quite large, however, and arranging a
suitable overlay structure may become burdensome. Unless you
have very special requirements you should select the clustered
library.
9.5.2 Memory Resident RMS
RMS is provided in a special library which uses memory resident
overlays to keep the virtual program space requirements to a
minimum. When you use this form you actually link only a stub
routine into your program. This stub intercepts the calls to the
various RMS routines and causes the appropriate RMS segment to be
mapped into your program space.
The RMS code itself is stored in a special library which must be
independently installed on your system. Since you are only
linking to the RMS stub the link process itself is much faster.
This form of RMS is considerably faster than the disk overlaid
version since the RMS segments are shared between all
applications and are usually already in memory. All of the disk
overhead associated with the RMS overlays is eliminated.
Use of this library also has the advantage that there is only one
copy of the RMS code on the system. This results in a very great
savings of space required to store task images.
Finally, it means that the system manager can upgrade to a newer
version of RMS by simply replacing the common library. There is
9-57
no need to re-link all the application programs.
You select this form of RMS by including a LIBR declaration in
your CMD file and a pointer to one of the secondary RMS ODL files
in your primary ODL file. The RMS library will use two of the
eight memory segments your task is allowed, thus consuming 16KB
of address space, about what the full, disk-overlaid version
requires.
9.5.3 Clustered RMS
Most languages require a run-time library of some sort and, in
most cases, this library is similar to the memory resident form
of RMS. Since RMS requires 16KB and most of the language
libraries require a similar amount this would limit your program
space to 32KB total. It is possible, however, to cluster the
language and RMS libraries in such a way that they occupy the
same virtual addresses so that the available program space is
increased to 48KB.
You direct the task builder to cluster RMS and the language
library by replacing the LIBR statement in the CMD file with a
CLSTR statement. This is the default for most of the Build
options in the language processors.
9.5.4 Supervisor Mode RMS
Memory resident overlays are considerably faster than disk
overlays and they certainly reduce the amount of disk head
contention on the system, but they still consume CPU time. On
systems which support Supervisor Mode you may link RMS as a
supervisor mode library. This option will cause RMS to use two
otherwise idle Address Page Registers (APR) and will reduce the
time required for context switching between the RMS and language
libraries.
You specify supervisor mode RMS by changing the LIBR or CLSTR
statement in the CMD file to RESSUP and making certain explicit
references to special RMS modules in your ODL file. See the
RMS-11 User Guide for details.
9.6 Useful Options
The Task Builder offers several useful options to the developer.
These options are invoked with statements in the CMD file.
9.6.1 Privilege
In the early versions of RSX a privileged task was one which was
mapped directly to the executive and could read and write the
9-58
various data tables and structures. Privileged tasks are also
exempt from the system security mechanisms and can read, write,
modify and delete files regardless of ownership or protection
codes.
It is now possible to build a privileged task which does not map
to the executive but which can bypass file protection. You can
use this technique to permit a user to perform occasional
privileged operations. To Task Build a privileged program,
append the /PR:0 switch to the task filespec in the CMD file.
9.6.2 Task Priority
You may be able to improve the overall crispness of your
application by adjusting task priorities. You may modify a task
priority when you install it or when you run it, but you may wish
to set the normal operating priority with the Task Builder.
You may specify a task priority with the PRI=nnn option.
Priority may range from 1 through 250 with higher numbers
indicating higher priority. Normal system priority is 50.
9.6.3 Task Name
You can set the task name with the TASK=name option in the CMD
file. This is the most convenient way to set a task name to
other than the task file name. It is also a convenient way to
specify a prototype task name.
9.6.4 Cross Reference Map
You can direct the Task Builder to append a Cross Reference table
to the Map file. This table will show where all the global entry
points (usually subroutine names) are defined in your task and
where they are referenced.
This option is particularly useful when you are restructuring
your overlays or trying to locate typographical errors in CALL
statements. You request this table by appending the /CR switch
to the map filename in the CMD file. The Cross Reference
Facility (CRF) must be installed on your system for this to work
properly.
9-59
Chapter 10
Packaging Applications for RSX
There are two views of layered product installation: That of the
person performing the installation and that of the developer who
is constructing the distribution kit. The former is discussed in
the section on Getting Started with RSX. This section will
outline what a developer must do to package an application for
installation on a production system.
The layered product developer is responsible for creating the
distribution kit - media and documentation - that will enable the
system manager to install the software as quickly and easily as
possible, and with the desired level of functionality. The
kitting process takes two forms according to whether the target
system is Micro/RSX or M-PLUS.
10.1 Kitting for Micro/RSX
Creating a software option for Micro/RSX will be easier if the
developer understands something of the underlying architecture.
10.1.1 Micro/RSX Layered Product Installation Architecture
Micro/RSX was designed with the naive user in mind. A Command
procedure is supplied with the system which controls installation
and removal of all layered products. This means that the
developer is responsible for packaging the application in such a
way that the command procedure will operate correctly.
When the installation control file (OPTION.CMD) is invoked, it
will mount the media on the drive selected by the user and will
look in directory [0,0] for a file INSTALL.DAT. This file is
provided by the developer and contains certain information about
ALL the options on the media set including the name of the option
and a pointer to the installation parameter file for the layered
product.
The installation parameter file is also provided by the developer
and it contains information about the files that comprise the
application, where they go on the target system and whether they
are to be deleted when the option is removed. It also contains
directions for actions that must take place during installation,
during system startup, and during removal. There are conditional
mechanisms by which the installation can be modified if, for
example, floating point hardware is available or I/D space is
supported by the CPU.
10.1.2 Creating the Kit
10-60
The kit is nothing more than an image of the application as it
will reside on the target system. If a file name APPL.XYZ is to
be placed into directory [POOF] on the system then it is stored
in a directory with the same name on the kit. When the
application is installed, the appropriate directories are created
as needed.
The simplest case is where the distribution media is diskettes.
The developer initializes enough diskettes to contain the
product, creates directories and copies files into place, just as
they will be on the customer's system. The developer then
creates the INSTALL.DAT and installation parameter files and adds
them to the disk set.
Creating a kit for magtape distribution is approximately the same
except that instead of creating an image of the application on
disk, the files are distributed as a collection of backup
save-sets. The INSTALL.DAT and parameter files are placed in a
special save-set at the beginning of the tape. It is usually
easiest to develop and debug an installation using disks as media
and then convert the procedure to magnetic tape. The conversion
from disk to tape kits is largely mechanical.
10.1.3 Developer's Note
Underlying this automation is a database of interest to the
developer, especially during the debugging of the installation
procedures. A list of all the software options installed on a
system is kept in the file LB:[1,2]OPTIONS.DAT. You can examine
and modify this file with your favorite text editor. Each record
contains the name of an application and a pointer to the
installation parameter file for that application. If the
OPTIONS.DAT file is deleted or corrupted, the layered product
database is destroyed and must be rebuilt, usually by deleting
all the application files and starting over.
Under certain conditions (noted below) OPTIONS.DAT is examined
during system startup and the various parameter files are opened
and scanned for any actions which must be performed. The
scanning process is slow, so as the actions are performed they
are also collected in a shortcut startup file
LB:[1,2]FASTART.CMD.
FASTART.CMD is the key to system startup. If it exists during
startup, it is executed directly, since doing so is much faster
than parsing the different layered product installation parameter
files.
If FASTART.CMD does not exist, then it is rebuilt from the
information in OPTIONS.DAT. Removing a software option from the
system is one of the ways in which this file might be deleted.
Since the parsing of the application installation parameter files
10-61
is slow, a system startup after an application has been removed
takes considerably longer than normal.
There are conditions, such as a product installation which
aborts, in which the contents of OPTIONS.DAT and FASTART.CMD no
longer match and the system may begin to malfunction. The cure
for this is to edit OPTIONS.DAT to assure that the contents are
as desired. Then delete all versions of FASTART.CMD and restart
the system. FASTART will be built anew during the subsequent
startup.
10.2 Kitting for M-PLUS
Creating a layered product for M-PLUS is largely a matter of
collecting all the components and creating a command file which
controls the installation procedure. The Indirect Command File
processor (IND) on RSX provides extensive and detailed access to
the underlying system and it is extremely rare that a layered
product has any installation requirements which cannot be
satisfied with IND. It is now fashionable to design the kit in
such a way that the installer need only copy the INSTALL.CMD file
into a temporary directory and invoke IND to execute it.
Writing IND procedures is no more difficult than other kinds of
programming but, like all implementation languages, it has a
flavor and style of it's own. If you have access to any layered
products you may wish to examine a copy of the INSTALL.CMD and
possibly even use it as a starting point.
When creating this command file the developer must consider the
spectrum of hardware and software configurations upon which the
product is to be supported. If any of the tasks require access
to the system executive and/or data base they will need to be
linked on the target system. Provide the required object modules
(in a library for convenience) and the command and odl files, and
link the task during installation.
The command file should be structured with as much of the dialog
as possible up front so that once it is complete the more lengthy
operations can proceed without further intervention. You should
provide for errors during installation and where it is not
possible to recover from the errors, provide a means by which the
installation can be unwound and the system restored to it's
original state.
Documentation for the manager should not only describe the
average conditions under which the product is supported but
should also explain enough of the overall architecture that a
knowledgeable system manager can tailor the product to an unusual
configuration. You should provide information about the
consequences of the different alternatives at each choice point.
You should also provide whatever information the manager will
10-62
need to modify the system startup file to create the proper
environment for your product. At the least, provide a system
startup command file fragment which can be inserted by the
manager.
Creating a disk kit is straightforward. Initialize the disk and
create as many directories as are required with whatever
structure is most convenient. Your command file can then copy
the files into the appropriate place on the target system.
Magnetic tape is a little trickier since it is a strictly
sequential medium and since there is a certain difficulty with
unusual types of files. The most commonly used magnetic tape
transfer utility is FLX which is rather like a PIP designed
specifically for magtape. Once your disk installation is working
to your satisfaction you can create a magtape kit by putting
together a command file which invokes FLX to copy all the
necessary files onto the tape, usually with the INSTALL.CMD file
going first. It is important to match the order of the files on
the tape with the order in which INSTALL.CMD attempts to copy
them onto the target system. If the files are in order (even if
some of them get skipped), the No-Rewind switch can be used with
FLX to greatly reduce the time spent searching along the tape for
the next file.
There are two problems which are frequently encountered with FLX.
The first is that there is no concept of file contiguity on
magnetic tape so contiguous files, particularly task images, must
be accorded special treatment. The FLX command to copy the task
image to the target disk must not only specify that the output
file be contiguous but must also specify an initial allocation.
This allocation must be the actual size of the image in blocks so
it will be necessary to update INSTALL.CMD anytime a task size
changes.
The other problem with FLX is that files with certain types of
contents, such as message files, cannot be transferred. The only
way around this is to convert the file to a simpler format before
building the kit, and transfer the source file and any necessary
commands or parameters for converting it back to the desired
format once it's on the target system. You can make use of the
temporary directory for such operations.
10.3 Kitting Applications for A-to-Z
A-to-Z was designed to be installed on Micro/RSX and has extended
the architecture somewhat to provide mechanisms by which a
layered product could update the A-to-Z database and integrate
itself into the A-to-Z environment.
A-to-Z has more recently been made available on Pregenned and
full M-PLUS and has carried with it a variation of the Micro/RSX
layered product installation. This makes it much easier for a
10-63
computer naive system manager to install and manage an
application mix. It also makes it easier on the product
developers since the kitting procedure for A-to-Z applications on
Micro/RSX and M-PLUS is the same. Only the media is different.
10-64
Chapter 11
System Operation
Once your application has been modified to take full advantage of
the features of RSX you may turn your attention to the details of
the underlying system. It is important to tune the application
first, since a job mix which misuses the underlying system or
which consumes an inordinate amount of resources will make it
difficult if not impossible to attain a satisfactory level of
performance no matter how much effort is put into the system.
This section describes the various dimensions along which you can
tailor RSX for a commercial application environment. This
section is not intended to be comprehensive but rather to get you
started as quickly as possible. Consult the RSX Command Language
Manual and the System Management Manual for more details.
11.1 Terminals
Traditionally, RSX is oriented towards expert operators who
require only the barest assistance or prompting from the Command
Line Interpreter. The terminal driver reflects this heritage and
developers who are moving to RSX from other operating systems may
be startled by some of the default terminal characteristics such
as being able to submit multiple commands for parallel processing
by the operating system.
Terminal characteristics are most commonly set during system
startup. On Micro/RSX and Pregenned M-PLUS the terminal
attributes may be set with suitable statements in SYSPARAM.DAT.
On M-PLUS you may include SET TERMINAL/attribute commands in
STARTUP.CMD.
You may also set terminal attributes as part of the user's
LOGIN.CMD. A-to-Z offers a comprehensive terminal management
architecture which you may wish to use in place of individual
system tailoring.
The following list of terminal attributes may be set or cleared
with the DCL SET TERMINAL/attribute command. This list is not
inclusive but contains the terminal settings which are typically
of importance on a commercial system. See the RSX Command
Language Manual for further details.
11.1.1 CLI
You may set or change the CLI attached to a terminal. Normally,
you set the default CLI (most commercial people prefer DCL) when
you create the account but there may be circumstances under which
you wish to change the CLI for some special purpose.
11-65
11.1.2 BROADCAST
Terminals which are set /BROADCAST will be subject to message
delivery when messages from a BROADCAST command arrive. This may
disrupt information on the screen but the message may be
important enough that this is acceptable. Certain messages, such
as those issued during the last five minutes of SHUTUP cannot be
intercepted. Setting of this attribute is left up to the
developer.
11.1.3 CONTROL=C
This attribute determines whether typing CTRL/C at the keyboard
aborts any program currently in progress at the terminal. If the
terminal is set /NOCONTROL=C typing CTRL/C will allow any ongoing
task to continue while the operator enters another command to the
CLI. This is the means by which you may invoke multiple tasks
simultaneously. The recommended setting for users in a
production environment is /CONTROL=C.
11.1.4 INQUIRE
Using the /INQUIRE attribute in a SET TERMINAL command will cause
the operating system to send the Request Device Attributes escape
sequence to the terminal and then to listen for an identification
string. If the response from the terminal is one of the DEC
supported terminals then the terminal type is set accordingly.
This attribute is useful when the terminal type on a particular
line might change from time to time such as when a LAN is in use.
You may override the terminal type setting at any time with the
SET TERMINAL/type command where type is one of VT100, VT200, etc.
11.1.5 FULL_DUPLEX
Terminals which are set /FULL_DUPLEX may accept input and receive
output at the same time and will allow overlapping terminal input
and output requests for languages which permit such operations.
The recommended setting is /FULL_DUPLEX.
11.1.6 ESCAPE
Setting a terminal /ESCAPE enables escape sequence processing.
Normally, an escape character in the input stream will be treated
as a terminator but if escape sequence processing is enabled any
characters which follow the escape will be examined. If the
characters form a proper escape sequence then the entire sequence
is passed as the input stream terminator. Some commercial
11-66
language runtime systems assert this attribute regardless of the
setting. The recommended setting is /ESCAPE.
11.1.7 EIGHTBIT
This attribute determines whether seven or eight bit bytes are
exchanged between the terminal and the operating system. If your
application makes use of Digital's Multinational Character Set
you should set the terminal /EIGHTBIT.
You may also wish to use this attribute if you have a modem
attached to a particular terminal line. Many of the asynchronous
file transfer protocol's require eightbit support. In this case
you must set the terminal characteristics in the system startup
command file.
11.1.8 HOSTSYNCH
The hostsynch attribute is used with block mode terminals or with
any device which may send a long string of characters to the
operating system. If hostsynch is enabled and if the typeahead
buffer is in danger of overflowing, the terminal driver will send
an XOFF character to the terminal. When the characters have been
processed and there is more room in the buffer, the driver will
send an XON character.
This attribute causes extra processing in the terminal driver and
should only be used when necessary. The recommended setting for
interactive terminals is /NOHOSTSYNCH.
11.1.9 SERIAL
Normally RSX permits parallel command execution. If a first
command typed at a terminal is followed by a second before the
first has completed, the two will execute in parallel. Setting
the terminal /SERIAL will cause the execution of the second
command to be postponed until the first has completed. The
recommended setting is /SERIAL.
11.1.10 SLAVE
A terminal which has the /SLAVE attribute will not accept any
input unless it has been attached by an application program.
This attribute is useful in situations where very inexperienced
operators are using the system. If such a user is running at an
unslaved terminal and the application should abort the user would
be confronted with the CLI and might do some inadvertent damage.
If the terminal is set /SLAVE, input characters will only be
accepted while the application is alive and well. If the program
11-67
aborts, the terminal will ignore any further input until it is
set /NOSLAVE from another terminal, or the system is restarted.
Note that the /SLAVE attribute should be set as part of the login
process and /NOSLAVE set as part of logout. A-to-Z automatically
sets terminals /SLAVE for all the user accounts.
11.1.11 LOWERCASE
Terminals set /LOWERCASE will echo characters just as they are
typed in. If the terminal is not set /LOWERCASE characters will
be converted to their uppercase equivalent before they are
echoed. /LOWERCASE is the recommended setting.
11.1.12 WRAP
Terminals which are set /WRAP will automatically wrap lines which
are longer than the line width setting. The terminal driver is
not quite smart enough to keep track of cursor positioning,
however, and if the terminal is set /WRAP, application screens
may be corrupted by the automatic wrapping. /NOWRAP is the
recommended setting.
11.1.13 TYPEAHEAD
Setting a terminal /TYPEAHEAD directs the operating system to
collect characters which are typed as input and to save them for
a subsequent read request by an application. If the terminal is
set /NOTYPEAHEAD, the characters will be discarded or will be
passed to the default CLI. /TYPEAHEAD is the recommended
setting.
On M-PLUS systems the command may be used to set the size of the
typeahead buffer as in SET TERMINAL/TYPEAHEAD:nn. If you are
using a block mode terminal or have other special requirements
you may wish to change the typeahead buffer setting from it's
default size of 86. This option is not available on Micro/RSX.
11.2 Partitioning Memory
The earliest versions of RSX required that the developer
determine the amount of memory available and divide it up into
fixed partitions. With the introduction of the System Controlled
partition (GEN) this is no longer necessary and most commercial
applications will require nothing more.
There are, however, three, comparatively simple, allocation
decisions remaining. Memory usage may be divided into four
general classes -- three of which the developer can allocate --
and the developer should assure that the memory allocated to each
class is sufficient to the task. The four classes are the
11-68
Executive, Secondary Pool, Task Space and the Disk Cache region.
You may have to add memory to your system to attain acceptable
performance.
11.2.1 Exec and Primary Pool
The RSX executive and it's dynamic memory region (Primary Pool)
are fixed in a special region during the system generation
process. The size of this region is fixed and pool space is set
to the highest possible value. There is little that you can do
to change this allocation.
11.2.2 Secondary Pool and Extension
Secondary Pool is another region of dynamic memory which is used
by the executive for a variety of purposes including task header
and logical name storage. The size of the Secondary Pool is
established during system startup.
Because Secondary Pool is used for so many different purposes it
is impossible for RSX to predict a suitable allocation figure
with any accuracy. You should take care to examine your
application in a production environment and assure that the
amount of memory allocated to Secondary Pool is adequate at peak
load. The RMD memory Display will indicate current utilization
and high water mark figures for Secondary Pool usage.
If Secondary Pool runs low or is exhausted, performance will
suffer and the system may malfunction. Excess pool allocation is
wasted memory. The default size of Secondary Pool is set during
system generation. You may increase the allocation by extending
the pool.
On Micro/RSX and Pregenned M-PLUS you extend Secondary Pool with
the SECONDARY_POOL statement in SYSPARAM.DAT. On M-PLUS you must
include LOA /EXP=SEC/HIGH/SIZE=nnn in your startup file. The
size parameter is the extension quantity expressed in words (not
bytes).
11.2.3 Task Space
Task space is the area of memory in which tasks, libraries,
common regions and loadable drivers reside. If there are more
tasks installed on the system than will fit in this region, some
of them will be moved to the checkpoint file on the disk.
Checkpointing active tasks against each other is a common source
of performance problems. You can use the RMD Memory display to
examine your system memory requirements under load. You will
need approximately 256KB for RSX plus 64KB for each interactive
user plus 64KB for each background processor or batch stream.
11-69
Tasks which have memory overlays require more memory and must be
included in these calculations on a case by case basis.
Space for tasks is not allocated but is, rather, what remains
after the executive, Secondary Pool and Cache allocations have
been made. Nevertheless, the amount of memory required for tasks
will affect the allocations you make for pool and cache.
11.2.4 Cache Region
When you have allocated sufficient Secondary Pool and have
determined how much memory you need for tasks, throw the
remainder into the Cache region. Cache is a means of
substituting (comparatively) cheap memory for much more
expensive, high performance disks. The bigger the cache is, the
better performance you will attain.
On Micro/RSX and Pregenned M-PLUS you set the size of the cache
region with the CACHE= statement. On M-PLUS you must insert a
SET CACHE command in your startup command file.
For hints on getting the most from you cache memory, see the
section on caching, later in this chapter.
11.3 Disks
One of the principal determinants of system performance is disk
I/O. Because the movement of the disk head and the rotation of
the platter are mechanical operations there are inherent delays
during which no processing can occur. This condition is further
aggravated on small system configurations where the number of
spindles is limited and the contention for head positioning is
intensified.
There are a number of system components which make demands on the
disk including system services, task activation and loading,
overlays and, of course, reading and writing data files.
Understanding each of these areas is important to understanding
the overall system throughput - moving the disk head away from a
data file to load an overlay segment from a task image incurs not
only the delay in loading the overlay but also the delay in
returning the head back to it's position over the data file.
Making the most efficient use of the available disk capacity on a
system requires you to have a basic understanding of the disk
structure On Disk Structure Level 1 (ODS-1) and the file system
(RMS). The principles described here pertain whether or not RMS
is used for record management. Furthermore, this is an aspect of
application development in which there is a great deal of
similarity between RSX and VMS. Any improvements made to the
performance of an application on RSX will also improve the
performance of the application on VMS.
11-70
11.3.1 Initializing
Before a disk can be used by RSX for any file structured
operation it must be initialized. Initialization is the process
of creating an empty ODS-1 directory structure on the disk such
that user files can be created and managed. The system disk is
usually initialized during installation of RSX. Any additional
disks must be initialized manually, an operation which can be
performed on-line.
11.3.1.1 Preparing a Disk
Before a disk can be initialized it must be formatted and scanned
for bad blocks. All of these operations require that you be
logged in as a privileged user and that the disk drive be
allocated as a private device. This will insure that no other
user is granted access to the device. Use the ALLOCATE command
and then mount the disk as a foreign file structure with the
MOUNT/FOREIGN command.
If the disk is a very old type, or if it was not manufactured by
DEC, you may need to format it. Formatting is the process of
writing sector header and address information on the disk
surface. Use the FMT utility. Most modern disks come from the
vendor already formatted.
Once the disk is formatted it should be scanned for bad blocks.
You can direct the BAD utility to write data patterns across the
disk and record which blocks on the disk fail to return the
information correctly. The location of the bad blocks is written
into a special area on the disk.
During initialization these blocks are collected together into a
special file (BADBLK.SYS). Since they are allocated to this file
they will not be used by the file processor for storing data.
Newer disks, such as the RDxx series, already have the bad blocks
information recorded on the disk volume.
11.3.1.2 Initializing a Volume
Use the INITIALIZE command to write the skeleton directory
structure to the disk. This operation creates the Master File
Directory, allocates file headers and creates a storage bitmap
for the disk. It also creates the bad block file and a null
checkpoint file. All these files are stored in a special system
directory ([0,0]) on the volume.
When you initialize a volume you must supply a label by which the
disk is identified during subsequent MOUNT operations. You also
specify a number of parameters which will affect it's subsequent
operation such as the maximum number of files and file headers,
11-71
the volume protection which determines who can access the disk as
a whole, the default protection and extension size for files
created on the volume.
The default values for each of these parameters are usually
adequate but there are occasional circumstances, possibly a
requirement to store a very large number of (small) files, where
you may wish to override the defaults with your own values.
11.3.2 Mounting
Before a disk can be used during a session it must be MOUNTed.
MOUNT is the procedure by which the disk is made known to the
operating system and the parameters regarding it's use are
determined. The system disk is mounted during system startup but
all other disks, be they physically removable or not, must be
mounted through commands. In the case of fixed disks which are
used in the day to day operation of the system the mount commands
are usually edited into the system startup command file.
When you mount a removable disk you should supply the volume
label which will be checked against the label on the disk itself.
You may also supply qualifiers with the MOUNT command which
determine whether the volume is to be public (available to all
users), shareable (available for MOUNTing by other users), or
private. Other qualifiers can be used to override the volume
protection, the default file protection and extension quantity,
and whether the files on the volume are to be cached in memory.
Once the disk has been mounted you may create directories with
the CREATE/DIRECTORY command, open and close files, read and
write data.
11.3.3 Dismounting
If a volume is removable, and you wish to exchange it for
another, you should DISMOUNT the disk before removal. Note that
you cannot dismount a disk completely if there are any files
still open. This can occur with a volume mounted SHAREABLE where
more than one program or user has files open.
If not all the files have been closed and not all users have
dismounted the disk the DISMOUNT command will only "mark" the
volume for dismount. The actual dismount will not occur until
the last active user dismounts the disk. Be careful if you are
allowing sharing of removable disks.
11.4 Creating Directories
All files are indexed in directories. A directory is a special
file which contains identification information for other files.
11-72
The Master File Directory (MFD) contains a list of User File
Directories (UFD) on a disk volume. A UFD contains a list of
user data files.
All directory files are created and maintained in a special
system area ([0,0]). There is one MFD and zero or more UFD's on
each disk. A directory contains a series of data records, each
containing the name of a file contained within the directory and
information about the location of the file header. The MFD
contains a list of all the UFD's on the volume. The UFD's
contain a list of the data files contained in the user directory.
When a file is to be opened the directory is determined, either
explicitly or through defaults, and the MFD is searched for the
proper UFD entry. The UFD is then located and it, in turn, is
searched for the file entry. The data in the entry is then used
to locate the file header in the volume index file. Note that
because the search through file entries is sequential the average
time to find a file will increase with the number of entries.
Collecting large numbers of files together into a directory will
slow down access time considerably.
Directory files have all the properties of data files in that
they have protection codes, can be copied, renamed and deleted.
The protection of the MFD is the volume protection, that is, it
determines which users can access the disk as a whole. If a user
is denied read access to the MFD then there is no way to get at
any of the UFD's and the data files. The protection of a UFD
determines which users can read and/or write the data files
listed within that directory.
Normally, the system area [0,0] is concealed and will not be
listed (such as in a DIRECTORY command) along with the other
directories on the disk. If you specify this area explicitly,
however, you can gain access to the MFD and the UFD's as though
they were data files which, in a sense, they are. An RSX expert
can create various kinds of magical effects and illusions such as
listing a file in multiple directories. But unless you consider
yourself to be such an expert, you should leave this area and
it's contents alone.
11.5 Files
A file consists of a header and a collection of extents. The
header is located in the volume index file ([0,0]INDEXF.SYS) and
contains information identifying the file, such as the file name
and type, the version, the owner ID, the creation date and the
most recent revision date. The location of the header block for
a file is reflected in the File ID which is a unique identifier
by which the file may be located and opened without searching
through the directory structures. Some RSX components manipulate
files by their ID's.
11-73
The header also contains a number of extent descriptors which
describe each of the segments of the disk which belong to the
file. Available disk blocks are listed in the disk bitmap file
([0,0]BITMAP.SYS). As a file is extended the header is updated
to include the new extents. If the extent descriptors become too
numerous to fit in a single header block, an extension header is
allocated. This is why the number of headers allocated for a
volume (declared during initialization) must always be greater
than the maximum number of files.
The number and the size of the extents which comprise a file will
determine the access speed. The extents may be physically
adjacent on the disk or they may be quite far apart. If the
extents are close together the disk heads will spend less time
when moving from one to another.
When a file is opened RSX will read the first few extent
descriptors into a memory structure called a Window. If a
request is made to read or write a logical block of the file
which lies outside of the window, RSX must return to the file
header (more disk head movement and further delay) to get a new
set of extent descriptors (a window turn). The more extents
there are in the header the more often RSX will have to disrupt
file operations to obtain new window data. RSX will normally try
to allocate the extents for a file as close together as possible
but the pattern of disk usage may preclude any real optimization.
11.5.1 File Performance
There is a tradeoff between file performance and disk space
utilization, that is, it is often possible to improve performance
of a file by 'wasting' a certain amount of space. There are
three actions you can take.
First, you can pre-allocate the file as a contiguous space.
Making the file contiguous will reduce the amount of disk head
travel and, equally important, will reduce the number of extents
required to describe the file. If a file is contiguous there
will be only a small number of extent descriptors and you may be
able to eliminate window turns altogether. The disadvantage is
that there may not be sufficient contiguous space on a disk and
once the space is allocated to a file it cannot be used for any
other purpose.
Second, pre-allocating a file will improve performance when the
file is first written whether or not the file is contiguous. By
pre-allocating the extents you eliminate the small extend
operations that would otherwise be required as the file grows.
Third, you can increase the size of the extension quantity or the
number of blocks that RSX will add to a file each time it is
extended. It is better (and possibly wasteful) to extend the
file less frequently and in larger chunks. Even if the file as a
11-74
whole is not contiguous it is nonetheless helpful if pieces of
the file are, since there will be fewer extent descriptors and
the blocks will be clustered.
11.5.2 File Protection
RSX supports the same, multilevel file protection that is used in
VMS. Potential users of a file are grouped into four categories:
Owner, Group, System, and World. Membership in these groups is
determined by UIC. The Owner of a file is a user with the same
UIC as that listed as the file owner in the header. A group
member is a user with the same group UIC code as the owner.
System is the operating system itself and any user with a
privileged account (a group number of 10 or less). World is
everyone else.
For each class of user you may specify four kinds of access.
Read access allows the user to read, copy or execute the file.
Write access allows the user to modify data within the file.
Extend access allows the user to increase the amount of space
allocated to the file. (In VMS, there is Execute access rather
than Extend access; "execute" is not a concept contained in RSX.)
Delete allows the user to delete the file altogether.
The protection of a file is determined when the file is created
or when it is set or changed with a SET PROTECTION command. A
hierarchy of defaults are in effect beginning with the system
default and continuing through the volume default and the user
default. Only the owner of a file or a privileged user can
change the protection of a file. Thus, it is possible for a
non-privileged user to set the protection of a file to prevent
inadvertent deletion by the system but not to prevent the system
or a privileged user from changing the protection to allow
subsequent deletion.
Note that the protection of a Directory determines which users
may access the files identifiers within the directory but not
necessarily the file data. You may allow another user access to
selected files within a directory or you may lock them out
altogether.
11.6 Logical Names
Older versions of RSX supported a very limited logical name
facility which could only be used with devices. Beginning with
V3.0, however, both Micro/RSX and M-PLUS support a VMS compatible
logical name facility. Names may be defined in User, Group and
System tables, may be nested, and may expand to either partial or
full filespecs.
Logical names may be established with the DEFINE command or with
calls to system services. Since DEFINE does not check the syntax
11-75
of the equivalence name it is possible to use logical names to
convey string information other than file specifications between
users and tasks on the system. This facility is used throughout
A-to-Z.
The logical name translation process follows the same rules on
RSX that it does on VMS. When a name is presented to the system
for translation the leftmost part is examined. If the leftmost
part is terminated with a colon or a space, an attempt is made to
translate it and if the translation succeeds, the equivalence
name is substituted and the translation is repeated with the new
name.
When the leftmost portion is terminated with anything except a
colon or space the translation process ceases and the resultant
name is returned to the caller or used as a file specification.
Thus, NAME or NAME:other will result in an attempt to translate
"NAME" whereas NAME.other or NAME;other will not.
Logical names may be defined with embedded colons but the rules
for translation become much more complicated and the beginner is
advised to avoid this practice.
Logical name tables are stored in the secondary pool. If you
plan to make extensive use of logical names you should use the
SHOW MEMORY command to see whether the pool is close to
exhaustion and more space should be allocated.
11.7 Disk Caching
Most applications will benefit from disk caching. Caching is
actually a means by which memory may be treated as a very fast
disk. Note that it is always better to eliminate the disk I/O
altogether or to eliminate as many sources of I/O as possible,
but once this has been accomplished you should enable caching.
To make use of caching you must first establish one or more cache
regions in memory and then notify the devices which are to use
them. You may do both of these with the MOUNT command. More
than one device may be associated with a region.
When you create the cache region you should specify as much
memory as you can afford to spare. Generally, the larger the
region is, the more effective the cache will be, but this must be
balanced against all the other demands made on the system memory.
In particular, if you reduce the memory available to tasks on the
system you may cause some degree of thrashing which would wipe
out all the benefits of caching and then some. There is no
particular advantage to allocating multiple regions unless you
have divided memory into multiple partitions.
When you mount a device and associated it with a cache region you
may specify the types of operations that are to be cached and the
11-76
maximum size in disk blocks for each type. Generally the
defaults are adequate but you may wish to increase the extent
size for VIRTUAL (application data) operations if you use files
with bucket sizes larger than five blocks. The extent size
should be at least as large as the bucket size or caching will
not take place for relative and indexed file operations.
If there are a large number of interactive users it might be
worth the additional expense of extra memory simply to increase
the size of the disk cache.
11.8 Printers and Queues
RSX offers full support for multiple Print and Batch Queues.
Queues are named and can be specific to a particular device or
generic for a class of devices. Included with this facility is
Transparent Spooling of physical devices which permits multiple
programs to access a single device simultaneously. There is also
support for application specific queues.
The Queue manager is also somewhat complex and can be confusing
to the neophyte. Names of devices, device processors and queues
can be hard to distinguish and the sequence of startup commands
on M-PLUS is complex. Nonetheless, this is a powerful facility
and is one of very great interest to a commercial application
developer.
For information about submitting jobs, see the Batch and Queue
Operations manual. For information about starting and stopping
the queues, see the System Management Guide.
11.8.1 Queue Mechanism
A Queue is a collection of jobs waiting for processing. A queue
may be specific to a device or it may be generic and will pass a
job to whichever device is available. The Queue Manager (QMG) is
a task which collects queue requests from PRINT and SUBMIT
commands and distributes the jobs to the various queues.
A Processor is a task which is assigned to a specific device.
Jobs are passed from a queue to a processor for execution. A
spooled device is considered to be the device itself along with
the associated processor. A Batch processor is not associated
with a physical device.
11.8.2 Submitting Jobs
You may submit a job to a queue with either the PRINT or SUBMIT
commands. Both commands allow a variety of qualifiers which
control the way the job is executed.
11-77
SUBMIT is used for batch jobs. You may submit a job to the
generic batch queue or to a particular queue by name.
PRINT is used for print jobs and also for queueing jobs to
application processors. You may submit a job to the generic
print queue or to a particular queue or device by name.
You may also use the PRINT command to direct a job to a
particular device or processor by specifying qualifiers such as
FORM or LOWERCASE. If you use such a qualifier, the job will be
directed to a processor which was initialized with the proper
attributes or, if none such exist, it will be held until one is
created.
There are also a variety of commands for controlling jobs already
in a queue. You may HOLD a job and then RELEASE it at some later
time. You may also DELETE a job after it has been queued.
11.8.3 Startup
Starting up Print and Batch queues on Micro/RSX and Pregenned
M-PLUS requires only that you edit SYSPARAM.DAT and include the
proper QUEUE_MANAGER, BATCH_PROCESSORS and PRINTER statements.
Getting everything going on M-PLUS is a little more difficult. A
prototype QMGSTART.CMD file is provided with the system which
will start up one print queue and one batch queue. If this is
not sufficient for your needs you must modify the skeleton
QMGSTART command file.
Making such a change will be easier once you understand the
general workings of the queueing mechanism. You must install and
start QMG. This also initializes the generic PRINT and BATCH
queues. Then install the QMG interface QMGPRT. Then, for each
device or batch queue you must initialize a queue, install and
initialize a processor task and assign the queue to the
processor.
Each device must have an associated processor which is named
after the device. Both print and batch processors are provided
with RSX which may be installed with the appropriate task name.
Each processor must have an associated queue which has the same
name as the processor. You may also initialize generic queues
which are assigned to more than one processor.
You may also install application processors and initialize the
associated queues. Implementing such a processor is a more
advanced aspect of application development on RSX and is
mentioned here only to indicate that such a capability is
available.
11.8.4 Shutdown
11-78
If your application or system makes frequent use of queues you
should assure that system shutdowns are conducted in an orderly
fashion. Information about jobs awaiting execution is maintained
in a file (SP0:[1,7]QUEUE.SYS) and if the system is shutdown
abruptly while processing is in progress it is possible that
output data may be lost.
RSX is provided with a system shutdown command file
(LB:[1,2]SHUTUP.CMD) which will, among other things, assure that
the system queues are shut down in such a way that they may be
restarted properly when the system comes back up. You should
assure that your operators use this command procedure or provide
the same functionality via an application program.
11.8.5 Transparent Spooling
One of the features of the RSX queue architecture is Transparent
Spooling. A device which is spooled may be accessed by more than
one program simultaneously. When a program Opens such a device,
usually for output, the file system passes the data to the
processor instead of directly to the device. The processor,
which has the same name as the device, accumulates the data in a
temporary file and when the program closes the device channel,
the data file is queued, along with all other jobs, for the
physical device.
You may output a file to a spooled device by COPYing the file to
the device name. The COPY command will queue the request as
though you had used a PRINT command.
The advantage of this is that multiple programs may perform
output to a device, such as a printer, without assigning the
device for exclusive use or even determining if it is in use.
The disadvantage of this is that the program cannot determine
whether output is going directly to a device or is being
accumulated in a file. Programs which generate very large
amounts of output may inadvertently fill up the disk.
Another problem with transparent spooling is that programs which
require specific device control, such as a word processor which
wishes to stop printing at the top of a page and await a user
command, cannot determine whether it actually owns the device or
not.
The solution for both of these problems is for the program to
suspend the queueing operations temporarily, perform the
operations and then restore the queue. You can do this directly
from your program by spawning DCL with the following series of
commands. This example assumes that the spooled device is a
printer.
STOP/QUEUE queuename. Following this command, jobs will no
11-79
longer be taken from the queue although they may still be added
to it.
STOP/PRINT processor_name/JOB_END. This command will direct the
processor associated with the printer to stop at the end of the
current job. The processor name is usually the same as the
printer name, such as LP2. Once the current print job is
complete the processor will release the printer.
Then, from your program, spawn DCL to ALLOCATE the printer. This
command will fail while the printer is still busy but will
succeed once it is free. You may then Open the printer and be
assured of full, direct control over the device.
When your program is finished with the device, you should restore
it to spooled status. Spawn DCL with a DEALLOCATE command to
release the device. Then spawn the START/PRINT processor_name
and START/QUEUE queuename commands to resume normal operations.
11-80
Chapter 12
Troubleshooting
The performance of a system as a whole is dependent on the amount
of resources required by an application and the efficiency with
which the available resources are managed. The tuning process
must begin by reducing the resources required by an application
to the absolute minimum. It is always better to eliminate the
need for a resource than it is to improve it's availability or
performance.
Once the application tuning is complete, you may begin
determining that the resources available on the system are
adequate to the demand and that they are being managed properly.
There is little that a well tuned system can do to improve the
performance of an application which is inefficient or is
consuming an inordinate number of resources.
On the other hand, there are some ways that a poorly tuned system
can slow things down. If the system is not managing available
resources well then performance will suffer. The tuning process
is one of identifying resources which are insufficient or which
are being managed poorly.
12.1 Before You Start
A common cause of system performance problems is insufficient
hardware. There may not be enough memory. The processor or
disks may be too slow.
Another possibility is that error rates on malfunctioning devices
may be impeding the application by causing excessive numbers of
retries. This is not to say that you should blame all your
problems on hardware but rather that you should not overlook the
obvious.
Once you are satisfied that the hardware is sufficient to the
task and is working properly, you should begin examining the way
in which it is being used.
12.2 Diagnostic Tools
There are three tools provided with RSX which can be used to
examine a system under load and which may point out which
resources are in short supply or where imbalances have occurred.
The use of these tools is not something that you would want to
leave to an inexperienced user, however, and may require that you
be on site.
12-81
12.2.1 Resource Monitor Display (RMD)
RMD is the single most useful tool for diagnosing system problems
and observing system operation in general. It is invoked from
DCL with the SHOW MEMORY command and has a variety of displays
which provide information about the behavior of the system and
the usage of various resources. If you are having a performance
problem, begin your diagnosis with RMD.
RMD has a number of different screens which present various sets
of information about the system. You can switch from one screen
to another as well as changing the rate at which the screens are
updated with new information. You can even invoke RMD over a
dial-up line to investigate problems at a remote site.
The Memory display is the first to appear and you may switch from
one display to another on the fly. See the chapter on RMD in the
System Management Guide for more information.
12.2.1.1 Memory
The Memory display is the most frequently used and contains a
great deal of information. The display portrays system memory
and shows all the tasks and regions in their approximate
locations. You can watch programs move in and out as terminals
proceed through an application.
The display also shows the current and "high water" usage
statistics for primary and secondary pool, the amount of free
space on each of up to four disks, the name of the task currently
executing, and a summary of how many tasks are in memory versus
how many are checkpointed to the disk.
The display also shows the latest sequence number being used by
the Error Logger. If you have enabled Error Logging on your
system and you see the sequence number changing frequently it may
indicate that your hardware is marginal.
12.2.1.2 Active Task Display
The Active Task display is simply a list of all the tasks which
are active on the system. Each task is listed by name along with
it's size, priority, and status information. This is, perhaps,
less useful to a commercial programmer than the others.
12.2.1.3 Task Display
The Task display shows the contents of the header for a
particular task on the system. The header is a region of memory
that RSX uses to maintain task context and contains various
information such as hardware register contents, priority and
12-82
status indicators.
You can use the task display to examine the operation of any
program on the system in some detail. You can watch the program
open and close files, enter and leave certain processing states
and change size. In particular, if you observe the list of files
in use by the program to be changing rapidly you may have a
problem with too frequent file opens.
12.2.1.4 I/O Counts Display
The I/O Counts display shows the I/O and error logging counts for
up to six mass storage devices. You can use this display to
examine the distribution and level of disk activity on your
system. You can also check for errors accumulating on a device.
If you notice that a particular device is showing a
disproportionate amount of activity you may need to redistribute
the load by moving programs or files from one device to another.
You should also study the individual activity level for each
drive. If a particular disk is showing anything over 20 I/O
requests per second, the device may be saturated.
12.2.1.5 System Statistics
The System Statistics display shows you a variety of general
information about the operation of your system. The information
ranges from the total number of user logins to the average number
of QIO's issued per second. You can obtain information about CPU
utilization percentage of memory and checkpoint space being used,
and the rate at which tasks are being checkpointed.
You should pay particular attention to memory utilization and
checkpoint frequency. It is acceptable and appropriate for an
inactive or suspended task to be checkpointed to disk, such as a
menu driver which is waiting for a selected application to
complete. This will free up system memory for tasks engaged in
active processing.
However, if the display shows that tasks are being checkpointed
on a more than occasional basis, it may be that active tasks are
being checkpointed against each other. This condition, called
thrashing, is the worst performance problem you can have on RSX
and must be corrected by decreasing the number of active tasks
(users) or increasing the amount of memory.
12.2.1.6 Cache Region Display
The Cache display shows general information about the disk
caching on your system. By comparing the total number of reads
and writes to the number of cache hits you can determine the
12-83
effectiveness of the cache. If the hit rate is below 50% then
you are not getting maximum benefit from the cache and you should
adjust the cache parameters for better performance.
You should also note the Cache Used statistics. If the
percentage of a region in use is very high or very low you should
adjust the size of the region accordingly.
12.2.1.7 Detailed Cache Statistics
The Detailed Cache statistics display shows the finer details of
usage of a particular cache region. This display contains more
information than is usually required by commercial developers
although it may be of use by an expert for very fine system
analysis and tuning.
12.2.2 Error Logging
RSX supports an extensive error logging facility which can be
used for detecting and recording errors occurring on the devices
and memory in your system. The error information is detailed and
is collected for both soft (recoverable) and hard
(non-recoverable) errors. The information collected will assist
service personnel in diagnosing and correcting hardware problems.
Intermittent errors may affect performance of the system. If you
are experiencing a performance problem you may use error logging
to either identify a specific problem or to eliminate hardware
errors as the source of your problem.
You must explicitly enable error logging by toggling the
ERROR_LOGGING statement in SYSPARAM.DAT on Micro/RSX and the
Pregenned M-PLUS, or by including the appropriate ELI statements
in your STARTUP.CMD file for M-PLUS. See the Error Logging
Manual for further details.
The error logging facility consists of four components. The
logging task itself, ERRLOG, accumulates error log packets,
generated by the executive when an error occurs, and collects
them in a log file, usually LB:[1,6]LOG.ERR. The user interface
(ELI) sends command packets to ERRLOG and is used to start, stop
and control logging activity. The reporting facility (RPT) is
used to analyze the information in the log file and format it for
use. The fourth component (CFL) is used for modifying the
logging facility, such as when a foreign device is added to the
system. It's use is not normally required.
You may use error logging for detailed diagnosis or, more simply,
as an early warning mechanism. Activity in the ERRSEQ field in
the RMD Memory display will indicate that errors are occurring.
Note that this field will not be updated unless logging is
explicitly enabled as described above.
12-84
Note also that if you have enabled logging, and errors are
occurring, there may be considerable growth in the log file. You
should check the size of this file from time to time to assure
that it isn't gobbling up your disk.
12.2.3 Resource Accounting
Resource Accounting provides a record of system usage. Data is
gathered for individual users and for the system as a whole. You
can use this accounting as part of your customer billing
operation but you can also use it to evaluate the operation of
your system. You can direct Resource Accounting to collect
information about disk activity and throughput, about the usage
patterns of individual users of your system or about particular
tasks on the system which are suspected of heavy resource usage.
Resource Accounting is automatically enabled on Micro/RSX and
Pregenned M-PLUS. On M-PLUS you must include the appropriate
START/ACCOUNTING statements in your STARTUP.CMD. See the
Resource Accounting chapter in the RSX System Management Guide
for further details.
Accounting works much like Error Logging, that is, packets of
information are generated by the executive and are collected in a
file by the accounting facility. You can enable and disable
accounting with DCL commands in order to collect information only
during times of special interest to you.
Once some data has been collected you may use the SHOW ACCOUNTING
command to examine and format the information. You may specify
that only statistics for a particular terminal or task be
reported.
Normally, Resource Accounting only collects information about the
system as a whole. You may optionally direct that statistics be
collected on each task. This may be useful in Before/After
comparisons of application modules. To enable task accounting
you must add the TASK:YES parameter to the START/ACCOUNTING
statement.
If you are using accounting you should take care that the
accounting file does not use too much space on your disk. When
accounting statistics for everything on the system are gathered,
the data can pile up pretty quickly. This is particularly true
if you are collecting task statistics. The default accounting
file is LB:[1,6]ACNTRN.SYS.
12.3 What to Look For
There are a number of symptoms which you should be looking for if
you suspect that your system is performing below it's capability.
12-85
Many of these conditions can be detected by using RMD and
interpreting the data in light of your knowledge of the
architecture of the application.
Once you have identified a problem area you will have to take
some corrective measure. Generally the nature of the symptom
will suggest one or more actions to take.
12.3.1 Disk Saturation
Probably the most common problem encountered on a small system is
saturation of the disks. Configurations are limited by price
sensitivity to small numbers of drives and the disk head becomes
a potential bottleneck.
Use the RMD I/O display to examine the QIO rate for each disk.
If you see that the rate is close to the maximum as determined by
average access times for the drive, then you have identified a
bottleneck. More than 20 operations per second for a particular
disk drive usually indicates saturation.
12.3.2 Insufficient Memory
Use the System Statistics display of RMD to determine the
percentage utilization of memory. Memory on the system is
specifically allocated for the system executive and the various
common, secondary pool and cache regions. Whatever memory
remains is available for tasks.
If the percentage utilization of memory is very high there is the
possibility that there are more active tasks than can fit and the
executive will begin freeing up memory by moving otherwise
runnable tasks to the checkpoint file. This condition is called
thrashing and it is the cause of extreme performance degradation.
You must consider the different demands that are being made on
system memory. There is little you can do about the size of the
executive and the common regions such as RMS and the language
processors. Secondary pool usage will vary from time to time but
a maximum usage can be established over time and any memory
allocated beyond the maximum is wasted.
Tasks which have been suspended until some external process
occurs may be safely checkpointed and require no memory until
they become active once again.
Most applications are designed with one interactive task for each
interactive user with perhaps a small number of background or
batch processes running concurrently. The architecture of the
PDP-11 generally limits the memory requirements of disk overlaid
tasks to 64KB for each user or process. Tasks which are memory
overlaid must be evaluated individually. It is usually
12-86
sufficient, therefore, to allocate about 64KB for each
interactive user and for each background process. All remaining
memory may be allocated to the disk cache region.
12.3.3 Unbalanced Load
While you are in the I/O display of RMD notice whether the disk
related QIO's are evenly distributed across all the drives. If
one drive shows a much greater level of activity than the others,
you should investigate the cause. There are many system
operations which require disk I/O and some of them, such as
program overlays or implicit spooling, may have escaped your
notice. You may be able to even out the load by simply moving
commonly used programs or libraries from one disk to another.
12.3.4 Insufficient Pool
Use the Memory display of RMD to assure that there is sufficient
free space in both the Primary and Secondary Pools. Primary pool
space is fixed during SYSGEN and is generally sufficient.
Secondary Pool, however, is used for a wide variety of purposes
and it may be running low.
Note that the percentage figures calculated for secondary pool
may not be correct if the pool has been extended. If you suspect
that the pool may be running short, increase the allocation.
12.3.5 File Contention
File or record contention is sometimes difficult to detect and
requires a knowledge of the underlying application. Excessive
contention can occur if the application has been converted from
CTS-300 or VMS without regard to the differences in the way RMS
treats access combinations on shared files.
12.3.6 Low Cache Hit Rates
Use the RMD Cache display to assure that the existing cache
regions are showing high hit rates and are being used to about
80% of their capacity. Cache regions should be allocated with
good measure but not to the extent of reducing memory available
to other system facilities.
If hit rates are low you should find out why and take whatever
corrective actions are necessary. It may be that you have
forgotten to enable caching for certain disks or have chosen the
wrong combination of operation types. The size of the region may
be too small to be effective. If the percent usage of a region
is high but the hit rate is low, try increasing the size of the
region.
12-87
It may also be that the default extent size for a particular I/O
operation type is too small. I/O requests which are larger than
the extent size will never be cached. This condition is reported
on the Detailed Cache Statistics display of RMD. If any of your
high activity files have bucket sizes greater than five blocks
you should increase the extent size for VIRTUAL operations to
equal that of your bucket sizes. It may also prove that program
overlays show a high Extent Too Big rate requiring an increase in
the extent size for OVERLAY operations.
12.3.7 Insufficient Checkpoint Space
Use the RMD System Statistics display to assure that there is
always space available in the system checkpoint files. If no
free checkpoint space can be found the system will lock up until
enough tasks exit to free up the required space.
12.3.8 Excessive File Opens
Use the RMD Task display to examine the major components of your
application while the system is under load. The display will
show which files are open and you may discover that the list of
files changes frequently. This means that files are being opened
and closed, and if these operations are occurring often it slows
down other disk processing.
12.3.9 Overloaded Directories
Use the DIRECTORY command to look at the distribution of files
across your directories and disks. The more files there are in a
particular directory the more expensive file opens become. If
you have one directory with a large number of files, consider
separating the files into multiple directories. Better yet, add
a disk drive to the system and redistribute the users and files
accordingly.
12.3.10 Jobs at Wrong Priority
You should check to see that all the tasks on the system are
running at the correct priority. If you have inadvertently set
the priority of a non-critical task higher than that of more
important programs, performance will suffer. You may be able to
increase the responsiveness of terminal interactive tasks by
increasing their priority somewhat.
12.3.11 Misuse of Batch
The scheduling of batch processing on RSX is extremely flexible.
12-88
You may be able to achieve considerable performance gains by
scheduling non-critical batch processing for off hours. You may
also be able to improve performance by using multiple batch
queues. You can use the SHOW QUEUE command to examine the
condition of the queues on your system.
12-89
Hitchhiker's Guide to RSX
To get an updated copy of this document, when it becomes
available, simply return this sheet with the requested
information to:
A-to-Z Base Product Marketing
MKO1-2/E25
Digital Equipment Corporation
Continental Boulevard
Merrimack, New Hampshire 03054
Name: ____________________________________________
Title: ____________________________________________
Company:____________________________________________
Street: ____________________________________________
City: ____________________________________________
State: ____________________________________________
ZIP: ____________________________________________
12-90