HLRN User Data Migration - SGI to Cray

Announcements and Information

sorted by: newest first

The collected documentation for the Cray system can be found here: Right

Datum: 22.08.2013 Zeit: 16:18:28 Status: gesendet Ziel: All Users Nachricht:

Extended Operation of HLRN-II at Hannover

14th Announcement, published on Aug 22 by mail and in the NewsCenter

https://www.hlrn.de/home/view/NewsCenter/NewsArchive#ID-2407

Dear HLRN-User,

we have news from the Chancellor of Leibniz University Hannover, that 
our request for additional funds for continued operation of HLRN-II at 
the Hannover site will be decided on in September 2013 and that we may 
continue operation of HLRN-II beyond August 31, 2013 until that 
decision is made.

Further news from the Lower Saxony Ministry of Science and Culture 
today suggests that our quest for additional funds has a significant 
chance to succeed.

S. Schulze-Kremer

Last reminder: HLRN-II End of Service

13th Announcement, published on Aug 22 by mail and in the NewsCenter

https://www.hlrn.de/home/view/NewsCenter/NewsArchive#ID-2403

The HLRN-II system (home and working file systems and 
the compute components) will be finally switched off

    in Berlin on August 26,
    in Hannover on August 31.

  --> As of today there are 4 days left until August 26. <--

Data in your home and working directories $HOME, $WORK and $WORK2 
will NOT be preserved. You are responsible for copying/moving your 
important data from these directories:

  - in Berlin to the new HLRN-III file systems, see
    https://www.hlrn.de/home/view/NewsCenter/NewsArchive#ID-2373 ,

  - in Hannover to the PERM archive and also to the new HLRN-III 
    file systems there, see 
    https://www.hlrn.de/home/view/NewsCenter/NewsArchive#ID-2380 .

Do NOT wait anymore, migrate your data now! Due to possible 
insufficient network bandwidth, file transfers in the last days 
may not succeed.

The future availability of the HLRN-III Cray XC30 system will be 
announced separately.

(wwb)

Reminder: Data migration to HLRN-III

12th Announcement, published on Aug 19 by mail

1. Today (Monday, Aug 19) at 10:00 the Berlin data migration server
   bmig4 will be taken offline. Please use the other migration 
   servers bmig1, bmig2, and bmig3.

2. The HLRN-II system (home and working file systems and the compute
   components) will be finally switched off 
- in Berlin on August 26,
- in Hannover on August 31. 

Data in your home and working directories $HOME, $WORK and $WORK2
will NOT be preserved. You are responsible for copying/moving your
important data from these directories:
- in Berlin to the new HLRN-III file systems, see
  https://www.hlrn.de/home/view/NewsCenter/NewsArchive#ID-2373
- in Hannover to the PERM archive and also to the new HLRN-III 
  file systems there, see
  https://www.hlrn.de/home/view/NewsCenter/NewsArchive#ID-2380

There is still a chance that Hannover may continue operation for some
time after the end of August, but as of now this is NOT guaranteed,
see https://www.hlrn.de/home/view/NewsCenter/NewsArchive#ID-2367

Do NOT wait anymore, start migrating now! Due to possible insufficient
network bandwidth, file transfers in the last days may not succeed.

   -->  There are 7 days left until August 26.  <--

(wwb)

Reminder: Data migration to HLRN-III

11th Announcement, published on Aug 14 by mail

1. Since today, 9:30 the Berlin data migration servers are 
   online again. Data migration can continue there.

2. The HLRN-II system (home and working file systems and the compute
   components) will be finally switched off 
- in Berlin on August 26,
- in Hannover on August 31. 

Data in your home and working directories $HOME, $WORK and $WORK2
will NOT be preserved. You are responsible for copying/moving your
important data from these directories:
- in Berlin to the new HLRN-III file systems, see
  https://www.hlrn.de/home/view/NewsCenter/NewsArchive#ID-2373
- in Hannover to the PERM archive and also to the new HLRN-III 
  file systems there, see
  https://www.hlrn.de/home/view/NewsCenter/NewsArchive#ID-2380

There is still a chance that Hannover may continue operation for some
time after the end of August, but as of now this is NOT guaranteed,
see https://www.hlrn.de/home/view/NewsCenter/NewsArchive#ID-2367

Do NOT wait anymore, start migrating now! Due to possible insufficient
network bandwidth, file transfers in the last days may not succeed.

   -->  There are 12 days left until August 26.  <--

(wwb)

Hannover: Data migration to HLRN-III

10th Announcement, published on Aug 06 by mail and in the NewsCenter

https://www.hlrn.de/home/view/NewsCenter/NewsArchive#ID-2380

The data migration servers (hmig1, hmig2, hmig3 and hmig4) are now
also available in Hannover. It is possible to login to the migration
servers from hg0, hg1 and hg2.

The data migration period starts immediately and will last at least
until August 31, 2013.

 * Delete all data, that is not needed anymore (e.g. because you
   have saved it to PERM or elsewhere before).

 * Move important data, that is not needed within the next few
   months to the archive filesystem PERM.

 * For data, that is really needed immediately when the HLRN-III
   Cray system comes online, login to the migration servers and

   - copy the remaining data from the home directory to the new
     home directory ($NEWHOME)

   - copy the remaining data from the working directories to the
     new working directory ($NEWWORK)

The $NEWWORK filesystem in Hannover is not the final Lustre 
filesystem, but it will be possible to copy the data as soon as
the HLRN-III is installed.


Sincerely, your HLRN Team

Reminder: Data migration to HLRN-III

9th Announcement, published on Aug 05 by mail

The HLRN-II system (home and working file systems and the compute components) will be finally switched off at the end of August.

Data in your home and working directories $HOME, $WORK and $WORK2 will NOT be preserved. You are responsible for copying/moving your important data in these directories: - in Berlin to the new HLRN-III file systems - in Hannover to the PERM archive and very soon also to the new HLRN file systems there (separate announcement will follow).

See also our announcement at https://www.hlrn.de/home/view/NewsCenter/NewsArchive#ID-2373

There is still a chance that Hannover may continue operation for some time after the end of August, but as of now this is NOT guaranteed, see https://www.hlrn.de/home/view/NewsCenter/NewsArchive#ID-2367

Do NOT wait anymore, start migrating now! Due to possible insufficient network bandwidth, file transfers in the last days may not succeed.

(wwb)

Berlin: Data migration to HLRN-III starts

8th Announcement, published on Jul 29 by mail and in the NewsCenter

https://www.hlrn.de/home/view/NewsCenter/NewsArchive#ID-2373

Starting immediately!

The home and the working file systems will be switched off together with the compute components of the HLRN-II. Data in your home and working directories $HOME, $WORK and $WORK2 will not be preserved. You are responsible for copying/moving your important data in these directories to the new HLRN-III file systems for later use. For this purpose, we offer an immediately starting

data migration phase until August 26, 2013

Use our migration servers to copy all desired data from the old HLRN-II filesystems to the new Berlin HLRN-III directories. The important steps for data migration are:

On the HLRN-II SGI system in Berlin, examine your home and working directories:

  • Delete all data, that is not needed anymore (e.g. because you have saved it to PERM or elsewhere before).
  • Move important data, that is not needed within the next few months to the archive filesystem PERM
  • For data, that is really needed immediately when the HLRN-III Cray system comes online, login to the migration servers and
    • copy the remaining data from the home directory to the new home directory
    • copy the remaining data from the working directories to the new working directory
For more details and hints regarding the data migration please see our data migration online document.

Start now! Due to possible insufficient network bandwidth, file transfers in the last days may not succeed.

(wwb)

Berlin: 4th update on the transition to HLRN-III: further delays

7th Announcement, published by mail on Jul 26

The following information pertains to the Berlin HLRN complex.

Cray is still resolving unexpected problems with the setup of the filesystems for the data migration.

This forces us to postpone again the begin of the data migration until

Tuesday, July 30, 2013

at the earliest.

In addition, please note the previously announced maintenance in Berlin on Monday, July 29: https://www.hlrn.de/home/view/NewsCenter/NewsArchive#ID-2369

Please accept our sincere apologies for these repeated delays. We will announce the availability of the new file systems.

In preparation for the data migration we recommend the following actions, to be started immediately:

On the HLRN-II SGI system examine your home and working directories and 1. delete all files that are not needed anymore (e.g. because you have saved them to PERM or elsewhere before).

2. move important files that are not needed within the next few months to the archive filesystem PERM

You may also prepare checksums for your files on the SGI for later comparison with the checksums of the copies on the Cray file systems.

(wwb/tst)

Berlin complex: 3rd update on the transition to HLRN-III

6th Announcement, published by mail on Jul 22

The following information pertains to the Berlin HLRN complex.

The HLRN-III system vendor Cray has confirmed today to have the new file systems HOME and WORK ready for use by Thursday, July 25, 2013.

Therefore, we expect that our users can start the data migration to the new system on

Thursday, July 25, 2013, afternoon.

We will announce the availability of the new file systems.

In preparation for the data migration we recommend the following actions, to be started immediately:

On the HLRN-II SGI system examine your home and working directories and 1. delete all files that are not needed anymore (e.g. because you have saved them to PERM or elsewhere before).

2. move important files that are not needed within the next few months to the archive filesystem PERM

You may also prepare checksums for your files on the SGI for later comparison with the checksums of the copies on the Cray file systems.

(wwb/tst)

Update on Data Migration HLRN-II to HLRN-III in Hannover

5th Announcement, published by on Jul 22

https://www.hlrn.de/home/view/NewsCenter/NewsArchive#ID-2367

Dear HLRN-User,

since we are still awaiting the decision on the continued operation of HLRN-II in Hannover beyond its end of lifetime August 31, 2013, we here give an update on the data migration procedure for Hannover.

Data from your HOME directory in Hannover can be archived to PERM and from there later accessed from HLRN-III.

Data from your WORK directory can be migrated the same way. This may take some time if you plan to move large quantities of data. Hence we recommend you start archiving your data now, because in the worst case HLRN-II in Hannover has to go offline on 31.8.2013.

In addition, Cray plans to deliver and setup the future HOME appliance to Hannover beginning of August. We intend to connect it to HLRN-II so that you may use this file system as a fast temporary storage device for your HOME and WORK data. Again, if HLRN-II had to be shut down in Hannover on 31.8.2013, you might have only a few weeks time to copy your data this way.

So, in summary, please start migrating your data to PERM now, especially if you have large quantities of data. In addition, we intend to have the new HLRN-III HOME appliance up and running in Hannover beginning of August which from then on could be used as a fast temporary storage device for migration of your HOME and WORK data.

Sincerely, your Hannover HLRN Team

Berlin complex: 2nd update on the transition to HLRN-III

4th Announcement, published by mail on Jul 17

Due to delays in hardware shipment for the new file systems the data migration SGI -> Cray has to be started later than previously anticipated (July 16).

We now expect to have the final information on Monday, July 22, 2013 about the start of the migration. We will post an update as soon as the dates are fixed.

We apologize for the inconvenience.

Berlin complex: update on the transition to HLRN-III

3rd Announcement, published by mail on June 28

1. Data migration SGI -> Cray

In addition to our migration announcement https://www.hlrn.de/home/view/NewsCenter/NewsArchive#ID-2347 we now expect to have the new HOME and WORK file systems in Berlin in place already by July 16, 2013.

This will result in more time for our users to migrate (copy) their data in Berlin until the end of August 2013 from HLRN-II to HLRN-III.

Further updates and detailed migration instructions will follow as soon as they are available.

2. Reduced compute capacity

(see also https://www.hlrn.de/home/view/NewsCenter/NewsArchive#ID-2357 )

In preparation for the hardware installation of the upcoming HLRN-III system from Cray we have to shutdown three compute racks of the ice2 system on Monday, Jul 1, noon.

Therefore the compute capacity of the Berlin complex in the bigq class will be reduced from 960 ice2 nodes to 768 ice2 nodes until the end of lifetime of the system (end of August this year).

We apologize for the inconvenience.

HLRN-III Datenmigration

2nd Announcement, published in the June 2014 Newsletter (in German)

Das HLRN-II-System nähert sich dem Ende seiner geplanten Standzeit am 31. August 2013. Für Installation, Einrichtung und Bereitstellung der Ressourcen des HLRN-III ist folgender Ablauf vorgesehen.

Zeitplan für Berlin

Der Berliner Komplex wird endgültig am 31. August 2013 abgeschaltet. Es ist beabsichtigt, das HLRN-III-System der Firma Cray im Lauf des September schrittweise für den Nutzerzugriff zu öffnen. Dies bedeutet, dass es keinen Parallelbetrieb der Systeme von SGI und Cray geben wird. Zur Unterstützung der Migration von Nutzerdaten auf das Cray-System sollen die neuen HOME- und WORK-Dateisysteme bereits Ende Juli 2013 in Berlin in Betrieb genommen werden. Die Nutzer müssen dann bis Ende August ihre wichtigen Daten in Eigenregie auf die Cray-Dateisysteme migrieren (kopieren). Diese Migrationsphase muss für etwa eine Woche unterbrochen werden, in der die Cray-Dateisysteme in das parallel aufzubauende Computesystem integriert werden. Somit werden effektiv etwa drei Wochen Migrationszeit zur Verfügung stehen.

Zeitplan für Hannover

Die Installationsarbeiten in Hannover beginnen planungsgemäß nach Abschluss der Installation in Berlin. Dies bedeutet, dass die neuen HOME- und WORK-Dateisysteme in Hannover nicht bereits im August 2013 für die Datenmigration zur Verfügung stehen. Aus diesem Grund werden die Nutzer des Komplexes Hannover mit großen, zu migrierenden Datenmengen gebeten, ihre Daten ab sofort auf das dortige Archiv-System (PERM) zu migrieren (zu­sammengefasst in tar-Archiven). Die Daten können dann später von PERM in Hannover auf die Dateisysteme des HLRN-III geschrieben werden. Es ist geplant, den HLRN-III in Niedersachsen im Januar 2014 für den Nutzerbetrieb zu öffnen.

Wegen der Unterbrechung des produktiven Betriebs beim Übergang auf den HLRN-III und der notwendigen späteren Installation des HLRN-III in Niedersachsen, bemüht sich das Hannover-Team derzeit um eine Möglichkeit, den HLRN-II am Stand­ort Hannover über seine geplante Standzeit bis zum 31. August 2013 hinaus noch einige Monate für alle Nutzer in HLRN-Verbund weiter zu betreiben. Eine Entscheidung darüber wird in den nächsten Wochen erwartet.

Über den weiteren Fortschritt des HLRN-III-Projekts werden wir zeitnah informieren.

HLRN-III Migration Announcement

1st Announcement, posted on May 31, 2013

https://www.hlrn.de/home/view/NewsCenter/News#ID-2347

Dear HLRN User,

the HLRN-II system is approaching its planned end of lifetime on August 31, 2013.

The planned schedule for Berlin:

The Berlin complex will be finally shut down on August 31, 2013. HLRN and Cray intend to open HLRN-III for user access in September which means there will be no overlap of the SGI and Cray compute resources. In order to support the migration of user data to HLRN-III we intend to have the new home and work file systems in Berlin in place already by the end of July 2013. We would then ask you to migrate (copy) your data in Berlin during August 2013 from HLRN-II to HLRN-III. Detailed migration instructions will follow.

The planned schedule for Hannover:

The Hannover site is expected to continue operation of the SGI system until early 2014, but this depends on the availability of additional funds for operating costs past the anticipated lifetime, which at this time is not guaranteed. In the worst case, Hannover would also have to shut down on August 31, 2013.

Installation of the Hannover site will follow the installation in Berlin. This implies that Hannover might not have its new home and work file systems ready for data migration in August 2013. For that reason, we kindly ask you to start migrating your data (tar format) in Hannover to the Hannover perm file system immediately, in case you plan to migrate large portions of your work data.

HLRN and Cray intend to open HLRN-III for user access at the Hannover site at the beginning of 2014.

As with the previous systems, during the first weeks HLRN-III will open to registered HLRN users without NPL accounting.

We will keep you informed on further progress towards HLRN-III.

HLRN Technical Committee


Created by -- WolfgangBaumann - 10 Jun 2013
Last modification: WolfgangBaumann - 07 Feb 2014 14:26 / 4 years, 10 months, 1 week ago. (Version: 12)

 
Norddeutscher Verbund für Hoch- und Höchstleistungsrechnen
Back to top of page