US20010011324A1 - Method of data migration - Google Patents

Method of data migration Download PDF

Info

Publication number
US20010011324A1
US20010011324A1 US09/782,080 US78208001A US2001011324A1 US 20010011324 A1 US20010011324 A1 US 20010011324A1 US 78208001 A US78208001 A US 78208001A US 2001011324 A1 US2001011324 A1 US 2001011324A1
Authority
US
United States
Prior art keywords
access
storage system
data migration
server
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
US09/782,080
Other versions
US6374327B2 (en
Inventor
Hidetoshi Sakaki
Akira Kurano
Katsunori Nakamura
Takehiro Ishikawa
Toshiaki Hatanaka
Hiroshi Nishijima
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to US09/782,080 priority Critical patent/US6374327B2/en
Publication of US20010011324A1 publication Critical patent/US20010011324A1/en
Application granted granted Critical
Publication of US6374327B2 publication Critical patent/US6374327B2/en
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604Improving or facilitating administration, e.g. storage management
    • G06F3/0607Improving or facilitating administration, e.g. storage management by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/0647Migration mechanisms
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671In-line storage system
    • G06F3/0683Plurality of storage devices
    • G06F3/0689Disk arrays, e.g. RAID, JBOD

Definitions

  • the present invention relates to the process of data migration between storage systems. More particularly, the present invention relates to a method, apparatus and computer program for use in a system that performs data migration between storage systems for causing the system to accept access to the storage systems even though data migration between the storage systems has not been completed.
  • old volume indicates the resources of the system where the data originates during data migration.
  • new volume indicates the resources of the system where the data is destined during data migration.
  • Data migration is the transferring of data from the old volume to the new volume.
  • a volume is a storage system implemented by disk, memory circuits or the like.
  • IBM corporation developed a method of data migration which permits access from the CPU (disclosed by IBM 3990 model 6 Enhancements) using an extended remote copy function (hereafter XRC) or a peer to peer remote copy function (hereafter PPRC).
  • XRC extended remote copy function
  • PPRC peer to peer remote copy function
  • SDMS Symmetrix Data Migration Service
  • the system with XRC is provided with a function of storing data, to be written to the old volume (disk subsystem), from the CPU into a disk controller in the old disk subsystem.
  • the disk controller then stores the data into the old disk subsystem.
  • the CPU has the function of reading the stored data from the old disk subsystem via the disk controller. Thereafter, the CPU writes the data to the new disk subsystem thereby completing data migration.
  • the old disk subsystem and the new disk subsystem are connected to each other to permit communication between them.
  • data migration during access from the CPU is enabled.
  • access generated by execution of the job of a customer after the completion of data migration is prevented until the path to the disk subsystems has been switched.
  • intervention of the CPU to perform the access is not required.
  • the old and the new disk subsystems must be provided with the function of the PPRC.
  • the new disk subsystem processes the data with normal processing. Thus, access from the CPU during data migration is enabled.
  • the important feature of the function of the SDMS is that the old disk subsystem is not required to have the function of data migration.
  • the priority of the order of volumes that are to be transferred faster can be defined at the start of data migration.
  • dual operation of the new disk subsystem and an alternate disk subsystem can not be conducted unless repeated data migration processes are performed using a Symmetrix Remote Data Facility.
  • An object of the present invention is to provide a method, apparatus and computer program that allows for the safe writing of data by the CPU to the old and new volumes during data migration.
  • Another object of the present invention is to provide a method, apparatus and computer program for performing a data migration process while enabling dual operation of old and new volumes immediately after completion of data migration and enabling relatively immediate switching of the new volume to the old volume when the new volume failed during data migration.
  • Yet another object of the present invention is to provide a method, apparatus and computer program having a function of automatically adjusting data migration speed during data migration depending on the state of the load to the new volume so as to give priority to accesses to the new volume by the CPU.
  • Still yet another object of the present invention is to provide a method, apparatus and computer program for performing data migration in a manner that improves the performance of data migration and access to the volumes by the CPU.
  • the present invention provides a method, apparatus and computer program for performing data migration in a general purpose computer system.
  • the general purpose computer (information processing) system includes a central processing unit (CPU), a new disk controller (CU), a new disk volume (VOL), an old CU and an old VOL.
  • the new CU and new VOL are the destination of data migration and the old CU and the old VOL are the origin of data migration.
  • the new CU is provided with a data migration control part that controls data migration of data between the old and new VOLs and a cache that stores data for later storage to the new VOL.
  • a CU and a VOL can be implemented by a disk subsystem or a server.
  • the present invention provides a plurality of connections between the above-described elements. Particularly, the present invention provides a connection between the CPU and the new CU and a connection between the new CU and the old CU. There are also connections between the new CU and the new VOL and between the old CU and the old VOL.
  • connections between the CPU and the new CU and between the new CU and the old CU are configured to permit data migration from the old VOL to the new VOL.
  • data migration is started, access by the CPU to the old and new VOLs is temporarily stopped. After start of data migration, access from the CPU is permitted.
  • the data migration control part in the new CU judges if the access is to a region where data migration has been completed or to a region where data migration has not been completed.
  • the data exists in the new CU and the new CU responds to the access.
  • the data migration control part responds to the access by accessing the old CU through the connection between the old CU and the new CU to retrieve the data to the cache 18 in the new CU. After the CPU operates on the data, the data is then written to both the new CU and the old CU.
  • the data migration process can skip the tracks. Skipping tracks in the which data migration is not necessary improves the efficiency of the data migration process. Further, by leaving data of a track on the cache of the new CU allows for repeated access to the data by the CPU. Allowing for such repeated access to data in the cache improves the efficiency of access by the CPU during data migration.
  • the present invention adjusts data migration speed based on various information of the system that have been acquired and analyzed. Such information includes information of the utilization of the cache in the new CU, information of the utilization of the connection between the new CU and the old CU, information of the utilization of the old VOL and information of the utilization of data migration speed at the current speed.
  • the present invention provides a function where the data migration speed can be adjusted in a manner to give priority to access of the old VOL by the CPU.
  • FIG. 1 illustrates the configuration of the presented invention for performing data migration in a general purpose computer system
  • FIG. 2 is a flowchart of the operations of data migration performed by the data migration control part of the present invention
  • FIG. 3 is a flowchart of the operations performed by the present invention when access by the CPU occurs during data migration.
  • FIG. 4 is a flowchart of the operations performed by the present invention on the migrating side to adjust automatically data migration speed during data migration.
  • FIGS. 1 - 4 will be used to explain the features of the present invention for performing data migration in a general purpose computer system.
  • FIG. 1 illustrates the configuration of the present invention for performing data migration in a general purpose computer (information processing) system that is the preferred embodiment of the present invention.
  • the information processing system of the preferred embodiment of the present invention includes CPU 10 that is the central processing unit, a new disk controller unit (hereafter new CU) 11 and a new disk volume (hereafter new VOL) 12 that are the destination of data migration, and an old CU 13 and old VOL 14 that are the origin of data migration.
  • the new CU 11 is provided with a data migration control part 17 that controls data migration of the data and a cache 18 that stores data.
  • the new disk subsystem includes the new CU 11 and the new VOL 12
  • the old disk subsystem includes the old CU 13 and the old VOL 14 .
  • a CU and its VOL can be implemented by a server.
  • connections are provided for performing the data migration process of the preferred embodiment of the present invention.
  • the connections include a connection 15 between the CPU 10 and the new CU 11 , and a connection 16 between the new CU 11 and the old CU 13 .
  • connections 15 and 16 are configured to permit data migration from the old VOL 14 to the new VOL 12 .
  • the connections 15 and 16 are configured to permit data migration from the old VOL 14 to the new VOL 12 .
  • access by the CPU 10 to the old and new VOLs 14 and 12 is temporarily stopped.
  • Data migration from the old VOL 14 of the old CU 13 to the new VOL 12 of the new CU 11 is represented by the dashed-line arrow A.
  • access from the CPU 10 is restarted. Therefore, accesses to the new CU 11 during data migration by the CPU 10 can be performed.
  • the data migration control part 17 inside the new CU 11 judges if the access is to a region where data migration has not been completed or to a region where data migration has been completed.
  • the data exists in the new CU 11 and the new CU 11 performs the service as represented by the dashed-line arrow B.
  • data migration control part 17 performs the service by accessing the old CU 13 as represented by the dashed-line arrow C through connection 16 .
  • the relevant data is retrieved from the old CU 13 and stored in the cache 18 of the new CU 11 . Once the CPU 10 has operated on the data, the data is stored to both the new VOL 12 and the old VOL 14 . With these operations, access by the CPU 10 during data migration is enabled.
  • the interface between the new CU 11 and the old CU 13 through data migration control part 17 is controlled using the communication protocol of the CPU 10 , thereby not requiring any specific function for operating the old CU 13 .
  • FIG. 2 is a flow chart of the operations performed by the data migration control part 17 when data migration is conducted. Each of the steps of the flowchart in FIG. 2 can be implemented by the code of a computer program.
  • a pre-processing operation (Step 21 ) is performed where the execution condition of data migration is checked, the state of data migration is changed and other such pre-processing operations are performed.
  • migration management information used to manage data migration is initialized (Step 22 ).
  • the present invention may use either a copy pointer which indicates only the present position of data migration or a bit map that illustrates with respect to all tracks or cylinders whether data migration has been completed or not completed as the migration management information.
  • the migration management information is checked (Step 23 ). If data migration of all the data on the old VOL 14 has been completed then a post-processing operation is performed (Step 24 ). The post-processing operation includes operations such as changing the state of data migration. Thereafter, data migration is ended. If data migration of all of the data on the old VOL 14 has not been completed then a command chain for reading in multiple tracks next to the track (if it is a first copy, tracks at top region) where data migration has been completed is issued (step 25 ) to the old CU 13 based upon the migration management information.
  • the data migration control part 17 performs the emulation of issuing the command chain to the old CU 13 the same as if the command chain had been issued from the CPU 10 to the CU ( 11 ) as a Define Extent/Locate Record/Read Track) command chain. If the old CU 13 has a function equivalent to that of the new CU 11 , high speed transferring can be done with the dedicated command chain.
  • the data read in from the old CU 13 is stored to the cache 18 (step 26 ) inside the new CU 11 temporarily and the data in the cache 18 is written to the new VOL 12 (step 27 ).
  • the reading process from the old CU 13 is performed by sequential access. Since the old CU 13 is provided with a cache 18 same as the new CU 11 and a prefetch function, the data transferring from the cache 18 of the old CU 13 to the cache 18 of the new CU 11 can be conducted at high speed. Thus, the data stored in the cache 18 of the old CU 13 is in effect stored in the cache 18 of the new cu 11 (stage) and then destroyed when written to the new VOL 12 (destage).
  • This destaging is for the continuous tracks so that when the function of destaging a bundle of multiple continuous tracks is used, the efficiency of the destaging is improved and high speed data migration is enabled.
  • the path for the reading in of the data from the old CU 13 to the cache 18 of the new CU 11 and the path for destaging the data from the cache 18 to the new VOL 12 are different so that parallel execution of processings can be performed. Thus, the performance of the processing of data migration is improved. After data is written to the new VOL 12 processing is returned to Step 23 where the migration management information is checked.
  • FIG. 3 is a flowchart of operations performed when CPU access is generated during data migration. Each of the steps of the flowchart in FIG. 3 may be implemented by the code of a computer program.
  • Step 301 when access from the CPU 10 is generated a judgement is performed to determine whether the access is to a region where data migration has been completed or to a region where data migration has not been completed. This judgement as to whether access is to a region where data migration has been completed or a region where data migration has not been completed is performed based on the above-described migration management information.
  • the access from the CPU 10 is then judged to determine whether it is a READ or a WRITE access (Step 302 ). If the access is judged to be a READ access it is necessary to read in the data from the old CU 13 since the data does not exist in the new CU 11 .
  • connection 15 between the CPU 10 and the new CU 11 is disconnected (step 303 ) temporarily and the data migration control part 17 issues a command chain to the old CU 13 for reading the relevant tracks (step 304 ) through connection 16 .
  • the data migration control part 17 emulates the CPU 10 in that it issues a command chain similar to the command chain of Define Extent/Locate Record/Read track that is issued by the CPU 10 .
  • the tracks which have been read from the old CU 13 are stored to the cache 18 in the new CU 11 (step 305 ) and the channel between the CPU 10 and the new CU 11 is reconnected (step 306 ).
  • the data in the cache 18 is processed by transferring it to the CPU 10 (Step 307 ).
  • Step 308 If the access is to a region where data migration has not been completed and the access is judged to be a WRITE access (Step 302 ), processing proceeds to Step 308 .
  • the connection 15 between the CPU 10 and the new CU 11 is temporarily disconnected (Step 308 )
  • the old CU 13 is instructed to read in the relevant tracks (Step 309 ) through connection 16
  • the tracks which have been read in from the old CU 13 are stored to the cache 18 in the new CU 11 (Step 310 ) and the connection 15 between the CPU 10 and the new CU 11 is reconnected (Step 311 ).
  • Steps 308 through 311 can be replaced by an operation of writing data directly to the old CU 13 through connection 16 without performing the operations of reading data from the old CU 13 as per steps 308 to 311 .
  • connection 15 between the CPU 10 and the new CU 11 is disconnected (Step 313 ) temporarily and the data migration control part 17 issues a command chain for writing the data to the old CU 13 (Step 314 ) through connection 16 .
  • the data migration control part 17 emulates the CPU 10 by issuing a command chain similar to a write command chain issued by the CPU 10 .
  • the data to be written is transferred from the cache 18 of the new CU 11 to the old CU 13 through connection 16 (Step 315 ) and then the connection 15 between the CPU 10 and the new CU 11 is reconnected (step 316 ).
  • steps 305 and 310 by storing data from the CPU 10 to the cache 18 in the new CU 11 and writing the data stored in the cache 18 in the new CU 11 to the old VOL 14 and the new VOL 12 the present invention allows for data migration with respect to the data to be skipped. This is possible since the data has been moved from the old VOL 14 to the new VOL 12 and updated before writing therein. Further, the present invention allows for repeated access to data in the cache 18 of the new CU 11 by the CPU 10 .
  • a judgement must be performed to determine whether a region is a region to be skipped or not. This judgement can be performed by checking the migration management information when the migration management information is in the form of a bit map or a copy point. In order to improve efficiency a region in which data migration is to be skipped should have a considerable amount of range such as, for example, two cylinders.
  • the access is judged to be an access to a region where data migration has been completed and the access is a READ access, as the data exists on the new CU 11 , the data in the cache 18 of the new CU 11 or in the new VOL 12 is transferred to the CPU 10 as with general READ processing (Step 318 ) and the processing is completed.
  • the access is judged to be an access to a region where data migration has been completed and the access is a WRITE access, as with a general WRITE processing the data is transferred from the CPU 10 to the cache 18 of the new CU 11 (Step 319 ). Similar to the case where the WRITE access is to a region where data migration has not been completed, the connection 15 between CPU 10 and new CU 11 is temporarily disconnected (Step 320 ), and a command chain for writing the data to the old CU 13 through connection 16 is issued by the data migration control part 17 (Step 321 ). Thereafter the data in the cache 18 of the new CU 11 is transferred to the old CU 13 (Step 322 ) and the connection between the CPU 10 and the new CU 11 is reconnected (Step 323 ).
  • FIG. 4 is a flowchart which illustrates the operations performed for the automatic adjustment of migration speed during data migration. Each of the steps of the flowchart in FIG. 4 may be implemented by the code of a computer program.
  • Step 41 when conducting data migration, a processing to store information indicating the specified order of priority of data migration of the plural VOLs (Step 41 ) is required.
  • the queuing time of the command chains caused by contentions for the connection 16 between the new CU 11 and the old CU 13 the queuing time of the command chains caused by contentions for the connection 16 between the new CU 11 and the old CU 13 , the existence of the contentions for the same VOL by accesses from the CPU and the utilization of migration speed according to the current settings are measured (Step 42 ).
  • the processing of migration speed adjustment is, as shown in FIG. 4, performed at a set timing according to a timer.
  • information of data migration sequence of priority stored during the data migration process is acquired (Step 43 ).
  • cache resource information indicating the utilization of the cache 18 of the new CU 11 is acquired and compiled (Step 44 ).
  • the utilization factor of the cache is determined based on all processings in the new CU 11 utilizing the cache 18 including the processings related to data migration.
  • the utilization factor is calculated by counting the number of segments of the cache 18 that are not used.
  • path resource information indicating utilization of the path between the new and the old CUs 11 and 13 is acquired and complied (Step 45 ).
  • the average queuing time of the command chains for data migration caused by the generation of contentions for the path between new and old CU's 11 and 13 , measured during data migration, is used as the path resource information.
  • the rate of the change of the average value is calculated using past information.
  • old VOL resource information indicating utilization of the old VOL 14 is acquired and complied (Step 46 ).
  • the rate of the change of the existence of contentions is calculated using past information.
  • Step 47 Based on the order of the priority of sequence of data migration and the various resource information, a judgement is performed to determine whether migration speed should be changed (Step 47 ).
  • the migration speed is changed by manipulating at least one of the following two items. If the change is judged to be unnecessary, the processing is finished. If the change is judged to be necessary, the processing proceeds to the Step 48 .
  • the Step 48 there are two items to be adjusted. One of the two items is the number of tracks to be read in at one command chain for data migration. The other of the two items is the issuing interval of the command chains for data migration.
  • the various resource information indicates the tendency of an increase of access by the CPU, the number of the tracks to be read in is reduced or the issuing interval of the command chains issued by the CPU that have been given priority is lengthened.
  • the present invention allows for the setting of a VOL with higher priority such that migration speed is not reduced even when access by the CPU 10 is increasing.
  • the same data is stored on the new and the old volumes in the region where data migration has been completed.
  • immediate switching to dual operation using both old and new volumes is enabled after data migration has been completed.
  • faster switching to the old volume when failure has occurred in the new volume during the migration is enabled.
  • the migration speed can be automatically adjusted based on the priority order given the volumes.
  • faster migration of the volume with higher priority is enabled while giving priority to accesses from the CPU during data migration.
  • the data migration process can be skipped on tracks previously accessed by the CPU where data migration had not been completed since data migration is performed at the time of the previous access.
  • these accesses are processed with the cache of the new CU, thereby improving access performance of the CPU during data migration.

Abstract

A method, apparatus and computer program for controlling data migration in an information processing system which includes a central processing unit (CPU), a new storage system connected to the CPU and an old storage system connected to the new storage system. In the information processing system data migration is conducted to transfer data from the old storage system to the new storage system. The invention operates by permitting access by the CPU to the storage systems during data migration. When an access by the CPU is generated the invention determines whether the access is to a region where data migration has been completed or to a region where data migration has not been completed. If the access is to a region where data migration has been completed, then processing of the access is handled by the new storage system. If the access is to a region where data migration has not been completed, then processing of the access is handled by the old storage system causing data related to the access to be transferred from the old storage system to the new storage system. The speed of data migration can be adjusted based upon the utilization of the resources of the information processing system and information of the priority of access to the new storage system by the CPU.

Description

    BACKGROUND OF THE INVENTION
  • The present invention relates to the process of data migration between storage systems. More particularly, the present invention relates to a method, apparatus and computer program for use in a system that performs data migration between storage systems for causing the system to accept access to the storage systems even though data migration between the storage systems has not been completed. [0001]
  • Hereinafter the term “old” when used with other terms, for example, “old volume” indicates the resources of the system where the data originates during data migration. Further, hereinafter the term “new” when used with other terms, for example, “new volume” indicates the resources of the system where the data is destined during data migration. Data migration is the transferring of data from the old volume to the new volume. A volume is a storage system implemented by disk, memory circuits or the like. [0002]
  • During the process of data migration between the old volume and the new volume, where the CPU of the overall system reads data from the old volume and writes the data to the new volume, access by the CPU to volumes in response to a request generated by the execution of a job of a customer is prevented. Accesses to the volumes can sometimes be stopped for long time during data migration. Thus, the effectiveness of the overall system is reduced. [0003]
  • To address this disadvantage, IBM corporation developed a method of data migration which permits access from the CPU (disclosed by IBM 3990 model 6 Enhancements) using an extended remote copy function (hereafter XRC) or a peer to peer remote copy function (hereafter PPRC). This disadvantage was also addressed by EMC Corporation which developed a method of data migration that permits access from the CPU in Symmetrix Data Migration Service (SDMS) as described in the SYMMETRIX ICDA Family PRODUCT ANNOUNCEMENT SUMMARY, Nov. 6, 1995. [0004]
  • The system with XRC is provided with a function of storing data, to be written to the old volume (disk subsystem), from the CPU into a disk controller in the old disk subsystem. The disk controller then stores the data into the old disk subsystem. In order to accomplish data migration the CPU has the function of reading the stored data from the old disk subsystem via the disk controller. Thereafter, the CPU writes the data to the new disk subsystem thereby completing data migration. [0005]
  • After data migration has been completed, a request for access to the old disk subsystem generated by execution of a customer job is prevented until the path to the old disk subsystem is switched to that of the new disk subsystem. Access is then permitted to the new disk subsystem. [0006]
  • The above-described system with XRC requires that the function of the XRC be provided in the old disk subsystem and the CPU. Intervention of the CPU is not required to perform the access. However, as with general data migration, the new setting for the new disk subsystem is required for the CPU. [0007]
  • In the system with PPRC, the old disk subsystem and the new disk subsystem are connected to each other to permit communication between them. By writing data to be written by the CPU to the new disk subsystem through the connection, data migration during access from the CPU is enabled. As with the XRC, access generated by execution of the job of a customer after the completion of data migration is prevented until the path to the disk subsystems has been switched. In the system with PPRC, intervention of the CPU to perform the access, as with the system with the XRC, is not required. However, the old and the new disk subsystems must be provided with the function of the PPRC. [0008]
  • In the system with SDMS in order to conduct data migration, first the access from the CPU to the old disk subsystem is stopped. Then the connection of the access path from the CPU to the old disk subsystem is changed to the connection of the access path from the CPU to the new disk subsystem through a new access path between the old disk subsystem and the new disk subsystem. By reading data from the old disk subsystem and writing it to the new disk subsystem through the new access path, data migration is started. After the start of data migration, the access from the CPU is restarted. If the access from the CPU is to a region where data migration has been completed, the new disk subsystem processes the data directly. If the access from the CPU is to a region where data migration has not been completed, after the data of the relevant tracks in the old disk subsystem is read and written into the new disk subsystem, the new disk subsystem processes the data with normal processing. Thus, access from the CPU during data migration is enabled. [0009]
  • The important feature of the function of the SDMS is that the old disk subsystem is not required to have the function of data migration. The priority of the order of volumes that are to be transferred faster can be defined at the start of data migration. However, after completion of data migration dual operation of the new disk subsystem and an alternate disk subsystem can not be conducted unless repeated data migration processes are performed using a Symmetrix Remote Data Facility. [0010]
  • SUMMARY OF THE INVENTION
  • An object of the present invention is to provide a method, apparatus and computer program that allows for the safe writing of data by the CPU to the old and new volumes during data migration. [0011]
  • Another object of the present invention is to provide a method, apparatus and computer program for performing a data migration process while enabling dual operation of old and new volumes immediately after completion of data migration and enabling relatively immediate switching of the new volume to the old volume when the new volume failed during data migration. [0012]
  • Yet another object of the present invention is to provide a method, apparatus and computer program having a function of automatically adjusting data migration speed during data migration depending on the state of the load to the new volume so as to give priority to accesses to the new volume by the CPU. [0013]
  • Still yet another object of the present invention is to provide a method, apparatus and computer program for performing data migration in a manner that improves the performance of data migration and access to the volumes by the CPU. [0014]
  • The present invention provides a method, apparatus and computer program for performing data migration in a general purpose computer system. The general purpose computer (information processing) system includes a central processing unit (CPU), a new disk controller (CU), a new disk volume (VOL), an old CU and an old VOL. The new CU and new VOL are the destination of data migration and the old CU and the old VOL are the origin of data migration. The new CU is provided with a data migration control part that controls data migration of data between the old and new VOLs and a cache that stores data for later storage to the new VOL. A CU and a VOL can be implemented by a disk subsystem or a server. [0015]
  • The present invention provides a plurality of connections between the above-described elements. Particularly, the present invention provides a connection between the CPU and the new CU and a connection between the new CU and the old CU. There are also connections between the new CU and the new VOL and between the old CU and the old VOL. [0016]
  • The connections between the CPU and the new CU and between the new CU and the old CU are configured to permit data migration from the old VOL to the new VOL. When data migration is started, access by the CPU to the old and new VOLs is temporarily stopped. After start of data migration, access from the CPU is permitted. [0017]
  • In the present invention, upon issue of an access by the CPU the data migration control part in the new CU judges if the access is to a region where data migration has been completed or to a region where data migration has not been completed. When the access by the CPU is to a region where data migration has been completed, the data exists in the new CU and the new CU responds to the access. When the access by the CPU is to a region where data migration has not been completed, the data does not exist in the new CU. Thus, the data migration control part responds to the access by accessing the old CU through the connection between the old CU and the new CU to retrieve the data to the [0018] cache 18 in the new CU. After the CPU operates on the data, the data is then written to both the new CU and the old CU.
  • As per the above, in the present invention when the CPU accesses a region where data migration has not been completed, data from tracks read from the old volume are stored to the new volume and the data once operated on is stored to both the old volume and the new volume. Further, when the data is to be written to the new volume, the data is also written to the old volume. Thus, the new and the old volumes contain the same data at the regions where data migration has been completed. Therefore, immediate switching to dual operation after the completion of data migration is enabled. Also switching to the old volume is possible when a failure occurs in the new volume during data migration since the most recent update of data is reflected in the old volume. The above-described writing by the CPU to the new and old VOLs can be safely conducted during data migration. [0019]
  • By reflecting data of tracks read from the old disk subsystem during access by the CPU to a region where data migration has not been completed to the new volume, the data migration process can skip the tracks. Skipping tracks in the which data migration is not necessary improves the efficiency of the data migration process. Further, by leaving data of a track on the cache of the new CU allows for repeated access to the data by the CPU. Allowing for such repeated access to data in the cache improves the efficiency of access by the CPU during data migration. [0020]
  • The present invention adjusts data migration speed based on various information of the system that have been acquired and analyzed. Such information includes information of the utilization of the cache in the new CU, information of the utilization of the connection between the new CU and the old CU, information of the utilization of the old VOL and information of the utilization of data migration speed at the current speed. The present invention provides a function where the data migration speed can be adjusted in a manner to give priority to access of the old VOL by the CPU. [0021]
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • The present invention will be more apparent from the following detailed description, when taken in conjunction with the accompanying drawings, in which: [0022]
  • FIG. 1 illustrates the configuration of the presented invention for performing data migration in a general purpose computer system; [0023]
  • FIG. 2 is a flowchart of the operations of data migration performed by the data migration control part of the present invention; [0024]
  • FIG. 3 is a flowchart of the operations performed by the present invention when access by the CPU occurs during data migration; and [0025]
  • FIG. 4 is a flowchart of the operations performed by the present invention on the migrating side to adjust automatically data migration speed during data migration. [0026]
  • DESCRIPTION OF THE PREFERRED EMBODIMENT
  • FIGS. [0027] 1-4 will be used to explain the features of the present invention for performing data migration in a general purpose computer system.
  • FIG. 1 illustrates the configuration of the present invention for performing data migration in a general purpose computer (information processing) system that is the preferred embodiment of the present invention. The information processing system of the preferred embodiment of the present invention includes [0028] CPU 10 that is the central processing unit, a new disk controller unit (hereafter new CU) 11 and a new disk volume (hereafter new VOL) 12 that are the destination of data migration, and an old CU 13 and old VOL 14 that are the origin of data migration. The new CU 11 is provided with a data migration control part 17 that controls data migration of the data and a cache 18 that stores data. Here the new disk subsystem includes the new CU 11 and the new VOL 12, and the old disk subsystem includes the old CU 13 and the old VOL 14. In the present invention a CU and its VOL can be implemented by a server.
  • In the configuration connections are provided for performing the data migration process of the preferred embodiment of the present invention. The connections include a [0029] connection 15 between the CPU 10 and the new CU 11, and a connection 16 between the new CU 11 and the old CU 13. There is also a connection between the new CU 11 and the new VOL 12 and a connection between the old CU 13 and the old VOL 14.
  • A flowchart of the operation of the preferred embodiment of the present invention is explained below. The [0030] connections 15 and 16 are configured to permit data migration from the old VOL 14 to the new VOL 12. When data migration has been started, access by the CPU 10 to the old and new VOLs 14 and 12 is temporarily stopped. Data migration from the old VOL 14 of the old CU 13 to the new VOL 12 of the new CU 11 is represented by the dashed-line arrow A. After the start of data migration, access from the CPU 10 is restarted. Therefore, accesses to the new CU 11 during data migration by the CPU 10 can be performed.
  • Upon receipt of an access by the [0031] CPU 10 the data migration control part 17 inside the new CU 11 judges if the access is to a region where data migration has not been completed or to a region where data migration has been completed. When the access is to a region where data migration has been completed, the data exists in the new CU 11 and the new CU 11 performs the service as represented by the dashed-line arrow B. On the contrary, when the access is to a region where data migration has not been completed, the data does not exist in the new CU 11. Therefore, data migration control part 17 performs the service by accessing the old CU 13 as represented by the dashed-line arrow C through connection 16. The relevant data is retrieved from the old CU 13 and stored in the cache 18 of the new CU 11. Once the CPU 10 has operated on the data, the data is stored to both the new VOL 12 and the old VOL 14. With these operations, access by the CPU 10 during data migration is enabled.
  • The interface between the [0032] new CU 11 and the old CU 13 through data migration control part 17 is controlled using the communication protocol of the CPU 10, thereby not requiring any specific function for operating the old CU 13.
  • FIG. 2 is a flow chart of the operations performed by the data [0033] migration control part 17 when data migration is conducted. Each of the steps of the flowchart in FIG. 2 can be implemented by the code of a computer program. After the instruction of starting data migration a pre-processing operation (Step 21) is performed where the execution condition of data migration is checked, the state of data migration is changed and other such pre-processing operations are performed. Then migration management information used to manage data migration is initialized (Step 22). In order to manage the data migration process the present invention may use either a copy pointer which indicates only the present position of data migration or a bit map that illustrates with respect to all tracks or cylinders whether data migration has been completed or not completed as the migration management information.
  • After the above, the migration management information is checked (Step [0034] 23). If data migration of all the data on the old VOL 14 has been completed then a post-processing operation is performed (Step 24). The post-processing operation includes operations such as changing the state of data migration. Thereafter, data migration is ended. If data migration of all of the data on the old VOL 14 has not been completed then a command chain for reading in multiple tracks next to the track (if it is a first copy, tracks at top region) where data migration has been completed is issued (step 25) to the old CU 13 based upon the migration management information.
  • At this time, the data [0035] migration control part 17 performs the emulation of issuing the command chain to the old CU 13 the same as if the command chain had been issued from the CPU 10 to the CU (11) as a Define Extent/Locate Record/Read Track) command chain. If the old CU 13 has a function equivalent to that of the new CU 11, high speed transferring can be done with the dedicated command chain. The data read in from the old CU 13 is stored to the cache 18 (step 26) inside the new CU 11 temporarily and the data in the cache 18 is written to the new VOL 12 (step 27).
  • The reading process from the [0036] old CU 13 is performed by sequential access. Since the old CU 13 is provided with a cache 18 same as the new CU 11 and a prefetch function, the data transferring from the cache 18 of the old CU 13 to the cache 18 of the new CU 11 can be conducted at high speed. Thus, the data stored in the cache 18 of the old CU 13 is in effect stored in the cache 18 of the new cu 11 (stage) and then destroyed when written to the new VOL 12 (destage). This destaging is for the continuous tracks so that when the function of destaging a bundle of multiple continuous tracks is used, the efficiency of the destaging is improved and high speed data migration is enabled. The path for the reading in of the data from the old CU 13 to the cache 18 of the new CU 11 and the path for destaging the data from the cache 18 to the new VOL 12 are different so that parallel execution of processings can be performed. Thus, the performance of the processing of data migration is improved. After data is written to the new VOL 12 processing is returned to Step 23 where the migration management information is checked.
  • FIG. 3 is a flowchart of operations performed when CPU access is generated during data migration. Each of the steps of the flowchart in FIG. 3 may be implemented by the code of a computer program. [0037]
  • First, when access from the [0038] CPU 10 is generated a judgement is performed to determine whether the access is to a region where data migration has been completed or to a region where data migration has not been completed (Step 301). This judgement as to whether access is to a region where data migration has been completed or a region where data migration has not been completed is performed based on the above-described migration management information.
  • If the access is to a region where data migration has not been completed, the access from the [0039] CPU 10 is then judged to determine whether it is a READ or a WRITE access (Step 302). If the access is judged to be a READ access it is necessary to read in the data from the old CU 13 since the data does not exist in the new CU 11.
  • For this purpose, the [0040] connection 15 between the CPU 10 and the new CU 11 is disconnected (step 303) temporarily and the data migration control part 17 issues a command chain to the old CU 13 for reading the relevant tracks (step 304) through connection 16.
  • The data [0041] migration control part 17 emulates the CPU 10 in that it issues a command chain similar to the command chain of Define Extent/Locate Record/Read track that is issued by the CPU 10. The tracks which have been read from the old CU 13 are stored to the cache 18 in the new CU 11 (step 305) and the channel between the CPU 10 and the new CU 11 is reconnected (step 306). Then as with general cache READ hit processing, the data in the cache 18 is processed by transferring it to the CPU 10 (Step 307).
  • If the access is to a region where data migration has not been completed and the access is judged to be a WRITE access (Step [0042] 302), processing proceeds to Step 308. In this case, as with the READ access, the connection 15 between the CPU 10 and the new CU 11 is temporarily disconnected (Step 308), the old CU 13 is instructed to read in the relevant tracks (Step 309) through connection 16, the tracks which have been read in from the old CU 13 are stored to the cache 18 in the new CU 11 (Step 310) and the connection 15 between the CPU 10 and the new CU 11 is reconnected (Step 311). Then the data is transferred from the CPU 10 to the cache 18 in the new CU 11 (Step 312). Steps 308 through 311 can be replaced by an operation of writing data directly to the old CU 13 through connection 16 without performing the operations of reading data from the old CU 13 as per steps 308 to 311.
  • After the [0043] above connection 15 between the CPU 10 and the new CU 11 is disconnected (Step 313) temporarily and the data migration control part 17 issues a command chain for writing the data to the old CU 13 (Step 314) through connection 16. The data migration control part 17 emulates the CPU 10 by issuing a command chain similar to a write command chain issued by the CPU 10. The data to be written is transferred from the cache 18 of the new CU 11 to the old CU 13 through connection 16 (Step 315) and then the connection 15 between the CPU 10 and the new CU 11 is reconnected (step 316).
  • In steps [0044] 305 and 310, by storing data from the CPU 10 to the cache 18 in the new CU 11 and writing the data stored in the cache 18 in the new CU 11 to the old VOL 14 and the new VOL 12 the present invention allows for data migration with respect to the data to be skipped. This is possible since the data has been moved from the old VOL 14 to the new VOL 12 and updated before writing therein. Further, the present invention allows for repeated access to data in the cache 18 of the new CU 11 by the CPU 10. When data migration is conducted a judgement must be performed to determine whether a region is a region to be skipped or not. This judgement can be performed by checking the migration management information when the migration management information is in the form of a bit map or a copy point. In order to improve efficiency a region in which data migration is to be skipped should have a considerable amount of range such as, for example, two cylinders.
  • When an access by the [0045] CPU 10 is judged to be an access to a region where data migration has been completed at step 301 a judgement is performed to determine whether the access is a READ access or a WRITE access (Step 317).
  • If the access is judged to be an access to a region where data migration has been completed and the access is a READ access, as the data exists on the [0046] new CU 11, the data in the cache 18 of the new CU 11 or in the new VOL 12 is transferred to the CPU 10 as with general READ processing (Step 318) and the processing is completed.
  • If the access is judged to be an access to a region where data migration has been completed and the access is a WRITE access, as with a general WRITE processing the data is transferred from the [0047] CPU 10 to the cache 18 of the new CU 11 (Step 319). Similar to the case where the WRITE access is to a region where data migration has not been completed, the connection 15 between CPU 10 and new CU 11 is temporarily disconnected (Step 320), and a command chain for writing the data to the old CU 13 through connection 16 is issued by the data migration control part 17 (Step 321). Thereafter the data in the cache 18 of the new CU 11 is transferred to the old CU 13 (Step 322) and the connection between the CPU 10 and the new CU 11 is reconnected (Step 323).
  • According to the above, when the WRITE access is to the region where data migration has been completed or the WRITE access is to the region where data migration has not been completed, a WRITE process is performed of writing data from [0048] cache 18 of the new CU 11 to the old CU 13 (Steps 313 to 316 and Steps 320 to 323). Thus, the old VOL 14 is always stored with the latest data. Therefore, after data migration has been completed, dual operation of the new VOL 12 and the old VOL 14 can be performed immediately. Further, relatively immediate restart of operation on the side of the old CU 13 can be performed after the failure of the new VOL 12 during data migration. Still further, due to the operations described above a WRITE access can be safely conducted during data migration.
  • FIG. 4 is a flowchart which illustrates the operations performed for the automatic adjustment of migration speed during data migration. Each of the steps of the flowchart in FIG. 4 may be implemented by the code of a computer program. [0049]
  • In FIG. 4 when conducting data migration, a processing to store information indicating the specified order of priority of data migration of the plural VOLs (Step [0050] 41) is required. When processing an instruction for the old CU 13 to read in tracks of data, the queuing time of the command chains caused by contentions for the connection 16 between the new CU 11 and the old CU 13, the existence of the contentions for the same VOL by accesses from the CPU and the utilization of migration speed according to the current settings are measured (Step 42).
  • The processing of migration speed adjustment is, as shown in FIG. 4, performed at a set timing according to a timer. Thus, information of data migration sequence of priority stored during the data migration process is acquired (Step [0051] 43).
  • Then cache resource information indicating the utilization of the [0052] cache 18 of the new CU 11 is acquired and compiled (Step 44). The utilization factor of the cache is determined based on all processings in the new CU 11 utilizing the cache 18 including the processings related to data migration. The utilization factor is calculated by counting the number of segments of the cache 18 that are not used.
  • Then path resource information indicating utilization of the path between the new and the [0053] old CUs 11 and 13 is acquired and complied (Step 45). The average queuing time of the command chains for data migration caused by the generation of contentions for the path between new and old CU's 11 and 13, measured during data migration, is used as the path resource information. For the queuing time, the rate of the change of the average value is calculated using past information.
  • Then old VOL resource information indicating utilization of the [0054] old VOL 14 is acquired and complied (Step 46). The sum of the existence of contentions of command chains for data migration by accesses from the CPU 10 to the same VOL 12, 14, measured during data migration, is used as the old VOL resource information. For the sum of the existence of contentions, the rate of the change of the existence of contentions is calculated using past information.
  • Based on the order of the priority of sequence of data migration and the various resource information, a judgement is performed to determine whether migration speed should be changed (Step [0055] 47). The migration speed is changed by manipulating at least one of the following two items. If the change is judged to be unnecessary, the processing is finished. If the change is judged to be necessary, the processing proceeds to the Step 48. In the Step 48, there are two items to be adjusted. One of the two items is the number of tracks to be read in at one command chain for data migration. The other of the two items is the issuing interval of the command chains for data migration. When the various resource information indicates the tendency of an increase of access by the CPU, the number of the tracks to be read in is reduced or the issuing interval of the command chains issued by the CPU that have been given priority is lengthened.
  • The present invention allows for the setting of a VOL with higher priority such that migration speed is not reduced even when access by the [0056] CPU 10 is increasing.
  • By use of the present invention, writing of data to the old and new VOLs can be safely conducted during data migration. Also access by the CPU can be conducted with certainty to the VOLs even though data migration has not been completed. [0057]
  • Further, by use of the present invention, the same data is stored on the new and the old volumes in the region where data migration has been completed. Thus, immediate switching to dual operation using both old and new volumes is enabled after data migration has been completed. In addition, faster switching to the old volume when failure has occurred in the new volume during the migration is enabled. [0058]
  • Still further, by use of the present invention the migration speed can be automatically adjusted based on the priority order given the volumes. Thus, faster migration of the volume with higher priority is enabled while giving priority to accesses from the CPU during data migration. [0059]
  • Still further yet, by use of the present invention, the data migration process can be skipped on tracks previously accessed by the CPU where data migration had not been completed since data migration is performed at the time of the previous access. Thus, when the tracks are accessed repeatedly, these accesses are processed with the cache of the new CU, thereby improving access performance of the CPU during data migration. [0060]
  • While the present invention has been described in detail and pictorially in the accompanying drawings it is not limited to such details since many changes and modifications recognizable to those of ordinary skill in the art may be made to the invention without departing from the spirit and the scope thereof. [0061]

Claims (40)

We claim:
1. A method of controlling data migration in an information processing system which includes a CPU, a new storage system connected to said CPU and an old storage system connected to said new storage system, wherein data migration is conducted to transfer data from said old storage system to said new storage system, said method comprising the steps of:
permitting access to said storage systems by said CPU during data migration;
determining whether said access is to a region where data migration has been completed;
processing said access to said new storage system when said access is to a region where data migration has been completed; and
processing said access to said old storage system and transferring data related to said access from said old storage system to said new storage system when said access is to a region where data migration has not been completed.
2. A method according to
claim 1
, wherein each storage system is implemented by a disk subsystem which includes a disk controller and a disk volume.
3. A method according to
claim 2
, wherein said disk controller of said new storage system includes cache memory for temporarily storing said data related to said access transferred from said old storage system,
wherein said cache memory is accessible by said CPU.
4. A method according to
claim 1
, wherein said step of processing said access to said new storage system comprises the steps of:
checking whether said access is a READ access or a WRITE access;
when said access is a READ access, transferring data related to said access from said new storage system to said CPU; and
when said access is a WRITE access, transferring data related to said access from said CPU to both said new storage system and said old storage system.
5. A method according to
claim 1
, wherein said step of processing said access to said old storage system comprises the steps of:
checking whether said access is a READ access or a WRITE access;
when said access is a READ access, transferring data related to said access from said old storage system to said new storage system and transferring said data from said new storage system to said CPU; and
when said access is a WRITE access, transferring data related to said access from said old storage system to said new storage system, transferring data from said CPU to said new storage system so as to be combined with said data transferred from said old storage system and transferring said combined data to said old storage system.
6. A method according to
claim 1
further comprising the step of:
adjusting speed of data migration based on resource information acquired from said information processing system.
7. A method according to
claim 6
, wherein said resource information acquired from said information processing system includes cache resource information indicating utilization of cache memory included in said new storage system, path resource information indicating utilization of the connection between said new storage system and said old storage system and old storage system resource information indicating utilization of said old storage system.
8. A method according to
claim 7
, wherein said resource information further includes information concerning utilization of data migration speed according to current settings.
9. A method according to
claim 6
, wherein data migration speed is adjusted in a manner so as to give priority to accesses to said new storage system.
10. A method according to
claim 7
, wherein data migration speed is adjusted in a manner so as to give priority to accesses to said new storage system.
11. A computer program stored on a storage medium for controlling data migration in an information processing system which includes a CPU, a new storage system connected to said CPU and an old storage system connected to said new storage system, wherein data migration is conducted to transfer data from said old storage system to said new storage system, said computer program when executed causes said information processing system to perform the steps of:
permitting access to said storage systems by said CPU during data migration;
determining whether said access is to a region where data migration has been completed;
processing said access to said new storage system when said access is to a region where data migration has been completed; and
processing said access to said old storage system and transferring data related to said access from said old storage system to said new storage system when said access is to a region where data migration has not been completed.
12. A computer program according to
claim 11
, wherein each storage system is implemented by a disk subsystem which includes a disk controller and a disk volume.
13. A computer program according to
claim 12
, wherein said disk controller of said new storage system includes cache memory for temporarily storing said data related to said access transferred from said old storage system,
wherein said cache memory is accessible by said CPU.
14. A computer program according to
claim 11
, wherein said step of processing said access to said new storage system causes said information processing system to perform the steps of:
checking whether said access is a READ access or a WRITE access;
when said access is a READ access, transferring data related to said access from said new storage system to said CPU; and
when said access is a WRITE access, transferring data related to said access from said CPU to both said new storage system and said old storage system.
15. A computer program according to
claim 11
, wherein said step of processing said access to said old storage system causes said information processing system to perform the steps of:
checking whether said access is a READ access or a WRITE access;
when said access is a READ access, transferring data related to said access from said old storage system to said new storage system and transferring said data from said new storage system to said CPU; and
when said access is a WRITE access, transferring data related to said access from said old storage system to said new storage system, transferring data from said CPU to said new storage system so as to be combined with said data transferred from said old storage system and transferring said combined data to said old storage system.
16. A computer program according to
claim 11
, wherein said computer program when executed further causes said information processing system to perform the step of:
adjusting speed of data migration based on resource information acquired from said information processing system.
17. A computer program according to
claim 16
, wherein said resource information acquired from said information processing system includes cache resource information indicating utilization of cache memory included in said new storage system, path resource information indicating utilization of the connection between said new storage system and said old storage system and old storage system resource information indicating utilization of said old storage system.
18. A computer program according to
claim 17
, wherein said resource information further includes information concerning utilization of data migration speed according to current settings.
19. A computer program according to
claim 16
, wherein data migration speed is adjusted in a manner so as to give priority to accesses to said new storage system.
20. A computer program according to
claim 17
, wherein data migration speed is adjusted in a manner so as to give priority to accesses to said new storage system.
21. A server for controlling data migration with respect to another server in an information processing system which includes a CPU, wherein data migration is conducted to transfer data from said another server to said server, said server comprising:
a storage for storing data; and
a data migration control part for permitting access by said CPU to said servers during data migration, determining whether said access is to a region where data migration has been completed, processing said access to said server when said access is to a region where data migration has been completed, and processing said access to said another server and transferring data related to said access from said another server to said server when said access is to a region where data migration has not been completed.
22. A server according to
claim 21
, wherein each server is implemented by a disk subsystem which includes a disk controller and a disk volume as said storage.
23. A server according to
claim 22
, wherein said disk controller of said server includes cache memory for temporarily storing said data related to said access transferred from said another server,
wherein said cache memory is accessible by said CPU.
24. A server according to
claim 21
, wherein said data migration control part, when processing said access to said server, checks whether said access is a READ access or a WRITE access, transfers data related to said access from said server to said CPU when said access is a READ access, transfers data related to said access from said CPU to both said server and said another server when said access is a WRITE access.
25. A server according to
claim 21
, wherein said data migration control part, when processing said access to said another server, checks whether said access is a READ access or a WRITE access, transfers data related to said access from said another server to said server and transfers said data from said server to said CPU when said access is a READ access, and transfers data related to said access from said another server to said server, transfers data from said CPU to said server so as to be combined with said data transferred from said another server and transfers said combined data to said another server when said access is a WRITE access.
26. A server according to
claim 21
, wherein said data migration control part adjusts the speed of data migration based on resource information acquired from said information processing system.
27. A server according to
claim 26
, wherein said resource information acquired from said information processing system includes cache resource information indicating utilization of cache memory included in said server, path resource information indicating utilization of the connection between said server and said another server and another server system resource information indicating utilization of said another server.
28. A server according to
claim 27
, wherein said resource information further includes information concerning utilization of data migration speed according to current settings.
29. A server according to
claim 26
, wherein said data migration control part adjusts data migration speed in a manner so as to give priority to accesses to said server.
30. A server according to
claim 27
, wherein said data migration control part adjusts data migration speed in a manner so as to give priority to accesses to said server.
31. An information processing system which includes a CPU and a plurality of servers, wherein one of said servers controls data migration with respect to another of said servers, wherein data migration is conducted to transfer data from said another server to said one server, said one server comprising:
a storage for storing data; and
a data migration control part for permitting access by said CPU to said servers during data migration, determining whether said access is to a region where data migration has been completed, processing said access to said one server when said access is to a region where data migration has been completed, and processing said access to said another server and transferring data related to said access from said another server to said one server when said access is to a region where data migration has not been completed.
32. An information processing system according to
claim 31
, wherein each server is implemented by a disk subsystem which includes a disk controller and a disk volume as said storage.
33. An information processing system according to
claim 32
, wherein said disk controller of said one server includes cache memory for temporarily storing said data related to said access transferred from said another server,
wherein said cache memory is accessible by said CPU.
34. An information processing system according to
claim 31
, wherein said data migration control part, when processing said one access to said server, checks whether said access is a READ access or a WRITE access, transfers data related to said access from said one server to said CPU when said access is a READ access, transfers data related to said access from said CPU to both said one server and said another server when said access is a WRITE access.
35. An information processing system according to
claim 31
, wherein said data migration control part, when processing said access to said another server, checks whether said access is a READ access or a WRITE access, transfers data related to said access from said another server to said one server and transfers said data from said one server to said CPU when said access is a READ access, and transfers data related to said access from said another server to said one server, transfers data from said CPU to said one server so as to be combined with said data transferred from said another server and transfers said combined data to said another server when said access is a WRITE access.
36. An information processing system to
claim 31
, wherein said data migration control part adjusts the speed of data migration based on resource information acquired from said information processing system.
37. An information processing system according to
claim 36
, wherein said resource information acquired from said information processing system includes cache resource information indicating utilization of cache memory included in said one server, path resource information indicating utilization of the connection between said one server and said another server and another server system resource information indicating utilization of said another server.
38. An information processing system according to
claim 37
, wherein said resource information further includes information concerning utilization of data migration speed according to current settings.
39. An information processing system according to
claim 36
, wherein said data migration control part adjusts data migration speed in a manner so as to give priority to accesses to said one server.
40. An information processing system according to
claim 37
, wherein said data migration control part adjusts data migration speed in a manner so as to give priority to accesses to said one server.
US09/782,080 1996-12-11 2001-02-14 Method of data migration Expired - Lifetime US6374327B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US09/782,080 US6374327B2 (en) 1996-12-11 2001-02-14 Method of data migration

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
JP8-330883 1996-12-11
JP08-330883 1996-12-11
JP33088396A JP3193880B2 (en) 1996-12-11 1996-12-11 Data migration method
US08/988,979 US6230239B1 (en) 1996-12-11 1997-12-11 Method of data migration
US09/782,080 US6374327B2 (en) 1996-12-11 2001-02-14 Method of data migration

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US08/988,979 Continuation US6230239B1 (en) 1996-12-11 1997-12-11 Method of data migration

Publications (2)

Publication Number Publication Date
US20010011324A1 true US20010011324A1 (en) 2001-08-02
US6374327B2 US6374327B2 (en) 2002-04-16

Family

ID=18237582

Family Applications (2)

Application Number Title Priority Date Filing Date
US08/988,979 Expired - Lifetime US6230239B1 (en) 1996-12-11 1997-12-11 Method of data migration
US09/782,080 Expired - Lifetime US6374327B2 (en) 1996-12-11 2001-02-14 Method of data migration

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US08/988,979 Expired - Lifetime US6230239B1 (en) 1996-12-11 1997-12-11 Method of data migration

Country Status (4)

Country Link
US (2) US6230239B1 (en)
EP (1) EP0848321B1 (en)
JP (1) JP3193880B2 (en)
DE (1) DE69739041D1 (en)

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030225861A1 (en) * 2002-06-03 2003-12-04 Hitachi, Ltd. Storage system
US20040030837A1 (en) * 2002-08-07 2004-02-12 Geiner Robert Vaughn Adjusting timestamps to preserve update timing information for cached data objects
US20040068629A1 (en) * 2001-08-10 2004-04-08 Hitachi, Ltd. Apparatus and method for online data migration with remote copy
US20050096512A1 (en) * 2001-12-27 2005-05-05 Fox James K. System for monitoring physiological characteristics
US20050138308A1 (en) * 2003-12-17 2005-06-23 Noboru Morishita Remote copy system
US20050216532A1 (en) * 2004-03-24 2005-09-29 Lallier John C System and method for file migration
US7003706B2 (en) 2003-05-27 2006-02-21 International Business Machines Corporation Method, system, and program for improved device blocking and suspension
US20060085607A1 (en) * 2004-10-15 2006-04-20 Toshiyuki Haruma Method of introducing a storage system, program, and management computer
US7127581B2 (en) 2004-01-13 2006-10-24 Hitachi, Ltd. Data-migration method
US20070198604A1 (en) * 2006-02-21 2007-08-23 Hitachi, Ltd. Computer system, computer system management console, and data recovery management method
US20080181107A1 (en) * 2007-01-30 2008-07-31 Moorthi Jay R Methods and Apparatus to Map and Transfer Data and Properties Between Content-Addressed Objects and Data Files
US20080222218A1 (en) * 2007-03-05 2008-09-11 Richards Elizabeth S Risk-modulated proactive data migration for maximizing utility in storage systems
US8225036B2 (en) * 2007-07-24 2012-07-17 Hitachi, Ltd. Storage controller and method for controlling the same
US10664441B2 (en) 2016-07-15 2020-05-26 Fujitsu Limited Information processing system, information processing apparatus, and non-transitory computer-readable recording medium

Families Citing this family (74)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3287203B2 (en) 1996-01-10 2002-06-04 株式会社日立製作所 External storage controller and data transfer method between external storage controllers
JP3641872B2 (en) * 1996-04-08 2005-04-27 株式会社日立製作所 Storage system
JP3410010B2 (en) 1997-12-24 2003-05-26 株式会社日立製作所 Subsystem migration method and information processing system
US7114049B2 (en) * 1997-01-08 2006-09-26 Hitachi, Ltd. Adaptive remote copy in a heterogeneous environment
US6408369B1 (en) * 1998-03-12 2002-06-18 Emc Corporation Internal copy for a storage controller
US9361243B2 (en) 1998-07-31 2016-06-07 Kom Networks Inc. Method and system for providing restricted access to a storage medium
US7392234B2 (en) * 1999-05-18 2008-06-24 Kom, Inc. Method and system for electronic file lifecycle management
JP4689137B2 (en) * 2001-08-08 2011-05-25 株式会社日立製作所 Remote copy control method and storage system
JP3712874B2 (en) * 1998-10-29 2005-11-02 株式会社日立製作所 Data duplex control method
JP3250729B2 (en) * 1999-01-22 2002-01-28 日本電気株式会社 Program execution device, process movement method thereof, and storage medium storing process movement control program
US7167962B2 (en) 1999-08-19 2007-01-23 Hitachi, Ltd. Remote copy for a storage controller with reduced data size
US6463513B1 (en) * 1999-09-07 2002-10-08 International Business Machines Corporation Cache storage optimization in a data storage library of a redundant copy synchronization token tracking system
JP2002014777A (en) * 2000-06-29 2002-01-18 Hitachi Ltd Data moving method and protocol converting device, and switching device using the same
US7194590B2 (en) * 2001-02-28 2007-03-20 Hitachi, Ltd. Three data center adaptive remote copy
US7167965B2 (en) * 2001-04-30 2007-01-23 Hewlett-Packard Development Company, L.P. Method and system for online data migration on storage systems with performance guarantees
US7143252B2 (en) * 2001-05-10 2006-11-28 Hitachi, Ltd. Storage apparatus system and method of data backup
US20060195667A1 (en) * 2001-05-10 2006-08-31 Hitachi, Ltd. Remote copy for a storage controller with consistent write order
US7213114B2 (en) * 2001-05-10 2007-05-01 Hitachi, Ltd. Remote copy for a storage controller in a heterogeneous environment
US6832289B2 (en) * 2001-10-11 2004-12-14 International Business Machines Corporation System and method for migrating data
JP4168626B2 (en) * 2001-12-06 2008-10-22 株式会社日立製作所 File migration method between storage devices
JP4704659B2 (en) 2002-04-26 2011-06-15 株式会社日立製作所 Storage system control method and storage control device
JP2004102374A (en) * 2002-09-05 2004-04-02 Hitachi Ltd Information processing system having data transition device
JP2004110367A (en) * 2002-09-18 2004-04-08 Hitachi Ltd Storage system control method, storage control device, and storage system
US7263593B2 (en) 2002-11-25 2007-08-28 Hitachi, Ltd. Virtualization controller and data transfer control method
US6959370B2 (en) * 2003-01-03 2005-10-25 Hewlett-Packard Development Company, L.P. System and method for migrating data between memories
JP2004220450A (en) * 2003-01-16 2004-08-05 Hitachi Ltd Storage device, its introduction method and its introduction program
JP4267353B2 (en) * 2003-03-28 2009-05-27 株式会社日立製作所 Data migration support system and data migration support method
US7260739B2 (en) * 2003-05-09 2007-08-21 International Business Machines Corporation Method, apparatus and program storage device for allowing continuous availability of data during volume set failures in a mirrored environment
US7234077B2 (en) * 2003-06-24 2007-06-19 International Business Machines Corporation Rapid restoration of file system usage in very large file systems
JP2005018193A (en) * 2003-06-24 2005-01-20 Hitachi Ltd Interface command control method for disk device, and computer system
JP4349871B2 (en) * 2003-09-09 2009-10-21 株式会社日立製作所 File sharing apparatus and data migration method between file sharing apparatuses
JP4386694B2 (en) * 2003-09-16 2009-12-16 株式会社日立製作所 Storage system and storage control device
JP4598387B2 (en) * 2003-09-17 2010-12-15 株式会社日立製作所 Storage system
US7219201B2 (en) * 2003-09-17 2007-05-15 Hitachi, Ltd. Remote storage disk control device and method for controlling the same
JP4307202B2 (en) * 2003-09-29 2009-08-05 株式会社日立製作所 Storage system and storage control device
US20050083862A1 (en) * 2003-10-20 2005-04-21 Kongalath George P. Data migration method, system and node
JP4307964B2 (en) 2003-11-26 2009-08-05 株式会社日立製作所 Access restriction information setting method and apparatus
US7284017B2 (en) * 2003-12-29 2007-10-16 Storage Technology Corporation Data migration system and method
JP2005202893A (en) * 2004-01-19 2005-07-28 Hitachi Ltd Storage device controller, storage system, recording medium recording program, information processor, and method for controlling storage system
JP4391265B2 (en) 2004-02-26 2009-12-24 株式会社日立製作所 Storage subsystem and performance tuning method
US7149859B2 (en) * 2004-03-01 2006-12-12 Hitachi, Ltd. Method and apparatus for data migration with the efficient use of old assets
US7124143B2 (en) 2004-05-10 2006-10-17 Hitachi, Ltd. Data migration in storage system
JP4387261B2 (en) * 2004-07-15 2009-12-16 株式会社日立製作所 Computer system and storage system migration method
US7171532B2 (en) * 2004-08-30 2007-01-30 Hitachi, Ltd. Method and system for data lifecycle management in an external storage linkage environment
JP4646574B2 (en) 2004-08-30 2011-03-09 株式会社日立製作所 Data processing system
JP2006072440A (en) * 2004-08-31 2006-03-16 Hitachi Ltd Storage device and data transfer method
JP4498867B2 (en) * 2004-09-16 2010-07-07 株式会社日立製作所 Data storage management method and data life cycle management system
JP4438582B2 (en) * 2004-09-22 2010-03-24 株式会社日立製作所 Data migration method
US7620984B2 (en) * 2004-10-06 2009-11-17 Hewlett-Packard Development Company, L.P. Method of managing computer system
US20060075199A1 (en) * 2004-10-06 2006-04-06 Mahesh Kallahalla Method of providing storage to virtual computer cluster within shared computing environment
JP2006127028A (en) * 2004-10-27 2006-05-18 Hitachi Ltd Memory system and storage controller
JP2006260240A (en) * 2005-03-17 2006-09-28 Hitachi Ltd Computer system, storage device, computer software and data migration method
US7281104B1 (en) 2005-03-21 2007-10-09 Acronis Inc. System and method for online data migration
US7529903B2 (en) * 2005-07-05 2009-05-05 International Business Machines Corporation Systems and methods for memory migration
US7702851B2 (en) * 2005-09-20 2010-04-20 Hitachi, Ltd. Logical volume transfer method and storage network system
US20070106710A1 (en) * 2005-10-26 2007-05-10 Nils Haustein Apparatus, system, and method for data migration
JP4810210B2 (en) * 2005-12-06 2011-11-09 日本電気株式会社 Storage system, master storage device, remote storage device, data copy method, data copy program
US20070162691A1 (en) * 2006-01-06 2007-07-12 Bhakta Snehal S Apparatus and method to store information
US7603581B2 (en) * 2006-03-17 2009-10-13 International Business Machines Corporation Remote copying of updates to primary and secondary storage locations subject to a copy relationship
GB0606639D0 (en) 2006-04-01 2006-05-10 Ibm Non-disruptive file system element reconfiguration on disk expansion
JP4900784B2 (en) * 2006-04-13 2012-03-21 株式会社日立製作所 Storage system and storage system data migration method
JP4721057B2 (en) * 2006-06-28 2011-07-13 日本電気株式会社 Data management system, data management method, and data management program
JP2008027335A (en) * 2006-07-25 2008-02-07 Nec Corp Storage system, data migration method of storage system, and data migration device
JP4934820B2 (en) * 2007-05-29 2012-05-23 富士通株式会社 RAID device and stripe conversion control method for RAID device
US20100070722A1 (en) * 2008-09-16 2010-03-18 Toshio Otani Method and apparatus for storage migration
TWI392445B (en) * 2010-07-21 2013-04-01 Quanta Comp Inc Cooling apparatus for server rack
US8961278B2 (en) 2010-08-13 2015-02-24 Schneider Electric It Corporation Single rack cold air containment
JP2012133436A (en) * 2010-12-20 2012-07-12 Nec Corp Data storage device, data migration method and program
WO2012147135A1 (en) 2011-04-28 2012-11-01 Hitachi, Ltd. Data storage system and management method therefor
JP5657801B2 (en) * 2011-05-31 2015-01-21 株式会社日立製作所 Storage system and storage control method
JP5838652B2 (en) * 2011-08-23 2016-01-06 富士通株式会社 Data copy processing system
GB2504716A (en) 2012-08-07 2014-02-12 Ibm A data migration system and method for migrating data objects
CN111666036B (en) * 2019-03-05 2022-04-22 华为技术有限公司 Method, device and system for migrating data
JP7347007B2 (en) 2019-08-28 2023-09-20 富士フイルムビジネスイノベーション株式会社 Information processing device, information processing system, and information processing program

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06250795A (en) 1993-02-24 1994-09-09 Hitachi Ltd Disk array system
JP3246146B2 (en) * 1993-12-27 2002-01-15 株式会社日立製作所 Data transfer method for external storage device
JP2834399B2 (en) * 1993-12-28 1998-12-09 三菱電機株式会社 Duplex disk unit
US5537585A (en) * 1994-02-25 1996-07-16 Avail Systems Corporation Data storage management for network interconnected processors
US5680640A (en) * 1995-09-01 1997-10-21 Emc Corporation System for migrating data by selecting a first or second transfer means based on the status of a data element map initialized to a predetermined state
US5835954A (en) * 1996-09-12 1998-11-10 International Business Machines Corporation Target DASD controlled data migration move

Cited By (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040068629A1 (en) * 2001-08-10 2004-04-08 Hitachi, Ltd. Apparatus and method for online data migration with remote copy
US6938137B2 (en) * 2001-08-10 2005-08-30 Hitachi, Ltd. Apparatus and method for online data migration with remote copy
US20050096512A1 (en) * 2001-12-27 2005-05-05 Fox James K. System for monitoring physiological characteristics
US8560631B2 (en) 2002-06-03 2013-10-15 Hitachi, Ltd. Storage system
US20070283366A1 (en) * 2002-06-03 2007-12-06 Hitachi, Ltd. Storage System
US20030225861A1 (en) * 2002-06-03 2003-12-04 Hitachi, Ltd. Storage system
US7254620B2 (en) 2002-06-03 2007-08-07 Hitachi, Ltd. Storage system
US20040030837A1 (en) * 2002-08-07 2004-02-12 Geiner Robert Vaughn Adjusting timestamps to preserve update timing information for cached data objects
US6842825B2 (en) * 2002-08-07 2005-01-11 International Business Machines Corporation Adjusting timestamps to preserve update timing information for cached data objects
US7003706B2 (en) 2003-05-27 2006-02-21 International Business Machines Corporation Method, system, and program for improved device blocking and suspension
US7124267B2 (en) * 2003-12-17 2006-10-17 Hitachi, Ltd. Remote copy system
US20060277378A1 (en) * 2003-12-17 2006-12-07 Hitachi, Ltd. Remote copy system
US7506120B2 (en) 2003-12-17 2009-03-17 Hitachi, Ltd. Data transfer virtualization system
US20050138308A1 (en) * 2003-12-17 2005-06-23 Noboru Morishita Remote copy system
US7127581B2 (en) 2004-01-13 2006-10-24 Hitachi, Ltd. Data-migration method
US7536527B2 (en) 2004-01-13 2009-05-19 Hitachi, Ltd. Data-migration method
US7269703B2 (en) 2004-01-13 2007-09-11 Hitachi, Ltd. Data-migration method
US20070260840A1 (en) * 2004-01-13 2007-11-08 Naoki Watanabe Data-migration method
US20050216532A1 (en) * 2004-03-24 2005-09-29 Lallier John C System and method for file migration
US20060085607A1 (en) * 2004-10-15 2006-04-20 Toshiyuki Haruma Method of introducing a storage system, program, and management computer
US20070198604A1 (en) * 2006-02-21 2007-08-23 Hitachi, Ltd. Computer system, computer system management console, and data recovery management method
WO2008094594A2 (en) * 2007-01-30 2008-08-07 Network Appliance, Inc. Method and apparatus to map and transfer data and properties between content-addressed objects and data files
US20080181107A1 (en) * 2007-01-30 2008-07-31 Moorthi Jay R Methods and Apparatus to Map and Transfer Data and Properties Between Content-Addressed Objects and Data Files
WO2008094594A3 (en) * 2007-01-30 2009-07-09 Network Appliance Inc Method and apparatus to map and transfer data and properties between content-addressed objects and data files
US20080222218A1 (en) * 2007-03-05 2008-09-11 Richards Elizabeth S Risk-modulated proactive data migration for maximizing utility in storage systems
US7552152B2 (en) * 2007-03-05 2009-06-23 International Business Machines Corporation Risk-modulated proactive data migration for maximizing utility in storage systems
US8225036B2 (en) * 2007-07-24 2012-07-17 Hitachi, Ltd. Storage controller and method for controlling the same
US10664441B2 (en) 2016-07-15 2020-05-26 Fujitsu Limited Information processing system, information processing apparatus, and non-transitory computer-readable recording medium

Also Published As

Publication number Publication date
EP0848321A3 (en) 2005-05-18
JPH10171605A (en) 1998-06-26
EP0848321B1 (en) 2008-10-15
US6374327B2 (en) 2002-04-16
DE69739041D1 (en) 2008-11-27
EP0848321A2 (en) 1998-06-17
JP3193880B2 (en) 2001-07-30
US6230239B1 (en) 2001-05-08

Similar Documents

Publication Publication Date Title
US6230239B1 (en) Method of data migration
US6381677B1 (en) Method and system for staging data into cache
US7111134B2 (en) Subsystem and subsystem processing method
EP0354579B1 (en) A controller with a cache memory and control method of the cache memory
US7493450B2 (en) Method of triggering read cache pre-fetch to increase host read throughput
US8028127B2 (en) Automated on-line capacity expansion method for storage device
US6272571B1 (en) System for improving the performance of a disk storage device by reconfiguring a logical volume of data in response to the type of operations being performed
US5937427A (en) Information recording apparatus and method capable of efficient erase, write and verify processes
US6557079B1 (en) Remote data facility prefetch
US7171516B2 (en) Increasing through-put of a storage controller by autonomically adjusting host delay
US9063945B2 (en) Apparatus and method to copy data
EP1199637A2 (en) Disk controller comprising cache memory and method of controlling the cache
US20080320218A1 (en) Disk array apparatus and control method for disk array apparatus
JPH11282631A (en) Input and output control device and method
US5694570A (en) Method and system of buffering data written to direct access storage devices in data processing systems
US6658541B2 (en) Computer system and a database access method thereof
US5293618A (en) Method for controlling access to a shared file and apparatus therefor
US5426761A (en) Cache DASD sequential staging and method
US7099995B2 (en) Metadata access during error handling routines
US6487632B1 (en) Emulation technique for variable-length disk system to access data in a fixed-length disk system
JP3122702B2 (en) Disk device write-back control method
JPH1011337A (en) Method for controlling data in storage device
US6957302B2 (en) System and method for performing write operations in a disk drive using a write stack drive
US5860087A (en) Data search method wherein the court-key-data (CKD) storage subsystem disconnects from the central processing unit when searching
JPH04311216A (en) External storage controller

Legal Events

Date Code Title Description
STCF Information on status: patent grant

Free format text: PATENTED CASE

FEPP Fee payment procedure

Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Free format text: PAYER NUMBER DE-ASSIGNED (ORIGINAL EVENT CODE: RMPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

FPAY Fee payment

Year of fee payment: 4

FEPP Fee payment procedure

Free format text: PAYER NUMBER DE-ASSIGNED (ORIGINAL EVENT CODE: RMPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

FPAY Fee payment

Year of fee payment: 8

FEPP Fee payment procedure

Free format text: PAYER NUMBER DE-ASSIGNED (ORIGINAL EVENT CODE: RMPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

FPAY Fee payment

Year of fee payment: 12