Skip to content

How to move data from Puhti to Allas and vice versa?

There are several options on how to move your data between Puhti and Allas. This page gives you guidance for the data movement with clients a-commands, swift, rclone and s3cmd.

All the required packages and software needed for the clients are already installed on Puhti. To authenticate to a project you need to run command source /appl/opt/allas_conf.

Move data with a-commands

a-commands provides easy-to-use tools for basic usage of Allas. The functions to use for data movement between Puhti and Allas are:

  • a-put: Move data from Puhti to Allas
  • a-get: Move data from Allas to Puhti

Move data with Swift

Swift client provides functions upload and download for data movement:

swift upload <bucket name> <file name>

swift download <bucket name> <file name>
For more information, see Swift client.

Move data with rclone

Also, with client rclone you can move data between Puhti and Allas. Creating a bucket called 2000620-raw-data to Allas can be done with command:

rclone mkdir allas:2000620-raw-data

Uploading a file called file.dat to that bucket can be done with command rclone copy:

rclone copy file.dat allas:2000620-raw-data/

Downloading the file back to Puhti is done with the same rclone copy command:

rclone copy allas:2000620-raw-data/file.dat

Note: If you give a destination parameter name in the download command, rclone creates a directory where the download goes:

rclone copy allas:2000620-raw-data/file.dat doh

For further guidance for using rclone with Allas from Supercomputers, see Using Allas with rclone from Puhti.

Move data with s3cmd

For data movement between Puhti and Allas s3cmd provides functions:

  • put move data from Puhti to Allas

  • get: move data from Allas to Puhti

More examples of using s3cmd from Supercomputers can be found here.

Last update: September 4, 2020