Mass Data Handling

 

Mass data (experiment Raw Data and DST files) are stored on the IBM 3090 under MVS, the only operating system capable of holding Terabytes of data quasi-online. There are about 35 GBytes of temporary disk space available. The heart of the storage system is a Memorex Cartridge Robot (Automatic Tape Library, ATL) which holds at present some 5000 3480 cartridges. The overall capacity thus is about 1 Terabyte. The capacity will be increased in June 1993 to 4 Terabyte with higher density cartridges. The throughput will be improved by adding two more cartridge drives to the existing four. To import and export data from MVS refer to .

Data access is handled automatically by the MVS operating system. A subsystem, the Hierarchical Storage Manager (HSM) keeps track of a dataset's location (disk or tape cartridge) so one has not to worry about which disk or tape cartridge a dataset is on. Staging data from robot cartridge to disk is normally done automatically without user or operator intervention, 24 hours a day, seven days a week. However, in bottleneck situations cartridges with datasets that have not been touched for more than about half a year are likely to be exported outside the ATL. They can only be accessed during the operator's working hours (6:00-23:00). You can query the location of datasets under the FOPI account (KP09) with the TSO command (wildcards are allowed)

where <dataset-name>

If you need datasets to be ATL resident, you may use

where <dataset-name> INATL

which triggers the re-importing of cartridges into the ATL. Although normally not needed, you may stage datasets from cartridge to disk and back manually using the TSO commands

hrecall  <dataset-name> [ nowait ] # cartrigde->disk, optionally asynchronously
hmigrate <dataset-name> [ nowait ] # disk->cartrigde, optionally asynchronously

Data storing policies is currently as follows: Datasets with the second level qualifier (the name part after the user ID) being

• LMDV or DSTX

  are presumed to contain mass data like raw or DST files, respectively. They are allocated on the DATAxx disks. There is no quota on such files. They are migrated to the cartridge robot if they are not touched for a certain period and recalled automatically when needed. They will not be backed up.

  The FOPI naming convention is:

  • for Raw Data:

    KP09.LMDV.<exp-no>.RAW<run-no>
    e.g.:  
    KP09.LMDV.S018.RAW0815

  • for DST files:

    KP09.DSTX.<exp-no><generation>.DST<run-no>
    e.g.:  
    KP09.DSTX.S018C.DST0815

• RZFILE

  is defined for PAW RZ files (e.g. ntuples etc.) with a size of a few MB. They are allocated on USERxx disks and hence are backed up and contribute to your quota.

• RZDATA

  is reserved for large RZ files (large ntuples) of the order of 100 MB. Since such files are something like micro-DSTs they are treated like DSTX files (not backed up).