Steps to Migrate Data Between Linux Storage Devices

Table of Contents

1. Pre-Migration Checklist
2. Step-by-Step Migration Process
   • 2.1 Identify Source and Target Devices
   • 2.2 Prepare the Target Device
   • 2.3 Choose a Migration Method
   • 2.4 Execute the Migration
3. Post-Migration Verification
4. Troubleshooting Common Issues
5. Conclusion
6. References

Pre-Migration Checklist

Before starting, ensure you’ve covered these critical steps to avoid data loss or errors:

1. Backup Your Data

Always back up important data to an external device or cloud storage. Use tools like rsync, tar, or borgbackup to create a redundant copy. Example:

tar -czvf /backup/data_backup.tar.gz /path/to/source_data  

2. Check Device Health

Verify the source device isn’t failing (to avoid corrupted data transfers) and the target device is functional. Use smartctl (from smartmontools) for HDD/SSD health checks:

sudo smartctl -a /dev/sdX  # Replace /dev/sdX with your source/target device  

Look for “SMART overall-health self-assessment test result: PASSED.”

3. Ensure Sufficient Space

The target device must have enough free space for the source data. Check sizes with:

df -h /path/to/source  # Check source usage  
lsblk -o NAME,SIZE /dev/sdY  # Check target device size (replace /dev/sdY)  

4. Gather Tools

Install required tools (if missing):

• rsync: For file-level migration (most versatile).
• dd: For block-level cloning (bit-for-bit copies).
• parted/gparted: For partitioning the target device.
• e2fsprogs: For formatting (e.g., mkfs.ext4).

Install via:

sudo apt install rsync coreutils parted gparted e2fsprogs smartmontools  # Debian/Ubuntu  
sudo dnf install rsync coreutils parted gparted e2fsprogs smartmontools  # RHEL/Fedora  

Step-by-Step Migration Process

2.1 Identify Source and Target Devices

First, identify the source (device with data to migrate) and target (device to receive data) using Linux utilities.

List All Storage Devices

Use lsblk (list block devices) to view connected drives and partitions:

lsblk  

Example output:

NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT  
sda      8:0    0 465.8G  0 disk  
├─sda1   8:1    0   512M  0 part /boot/efi  
└─sda2   8:2    0 465.3G  0 part /  
sdb      8:16   0 931.5G  0 disk  # Target device (new SSD)  
└─sdb1   8:17   0 931.5G  0 part  # Target partition (to be mounted)  

• Source: Often mounted at / (root) or /mnt/source (if external).
• Target: Typically an unmounted disk (e.g., /dev/sdb) or partition (e.g., /dev/sdb1).

Confirm Device Paths

Double-check with fdisk to avoid mistakes (critical for tools like dd):

sudo fdisk -l /dev/sdX  # Replace /dev/sdX with suspected source/target  

2.2 Prepare the Target Device

If the target device is new or unformatted, partition and format it to match the source’s filesystem (e.g., ext4, XFS, Btrfs).

Step 1: Partition the Target (if needed)

Use parted (command-line) or gparted (GUI) to create partitions. For a single partition (common for data drives):

sudo parted /dev/sdY  # Replace /dev/sdY with target device (e.g., /dev/sdb)  

In the parted prompt:

• Set the partition table: mklabel gpt (for modern systems) or mklabel msdos (MBR, older systems).
• Create a partition: mkpart primary ext4 1MiB 100% (uses entire disk).
• Exit: quit.

Step 2: Format the Partition

Format the target partition with your desired filesystem (e.g., ext4):

sudo mkfs.ext4 /dev/sdY1  # Replace /dev/sdY1 with target partition (e.g., /dev/sdb1)  

Add a label (optional, for easier mounting):

sudo e2label /dev/sdY1 "MyData"  

Step 3: Mount the Target

Create a mount point and mount the target partition:

sudo mkdir -p /mnt/target  
sudo mount /dev/sdY1 /mnt/target  

Verify mounting with df -h:

df -h /mnt/target  

2.3 Choose a Migration Method

Linux offers multiple tools for data migration. Choose based on your needs:

2.4 Execute the Migration

Option 1: Migrate with rsync (Recommended for Most Users)

rsync is ideal for file-level migration, preserving permissions, timestamps, and symlinks. It also supports resuming interrupted transfers.

Command:

sudo rsync -av --progress --exclude="/mnt/source/lost+found" /mnt/source/ /mnt/target/  

Flags explained:

• -a: Archive mode (preserves permissions, timestamps, symlinks).
• -v: Verbose (shows progress).
• --progress: Displays transfer progress for individual files.
• --exclude: Skips unnecessary files (e.g., lost+found for ext4).

Notes:

• Trailing slashes matter! /mnt/source/ copies contents of source, while /mnt/source copies the folder itself.
• For network migration (e.g., to a remote server), add user@remote:/path/to/target.

Option 2: Clone with dd (Block-Level Copy)

dd creates an exact bit-for-bit copy of a disk or partition. Use this for cloning entire drives (e.g., system disks) or when you need an identical replica.

Command (clone entire disk):

sudo dd if=/dev/sdX of=/dev/sdY bs=4M status=progress  

Flags explained:

• if=/dev/sdX: Input file (source device).
• of=/dev/sdY: Output file (target device).
• bs=4M: Block size (4MB; larger = faster transfer).
• status=progress: Shows transfer speed and ETA.

Warning: Double-check if and of! Swapping them will overwrite your source data.

For partitions only: Replace if=/dev/sdX with if=/dev/sdX1 (source partition) and of=/dev/sdY1 (target partition).

Option 3: Simple Copy with cp (Small Datasets)

Use cp for small, non-critical data (e.g., documents, photos).

Command:

sudo cp -r /mnt/source/* /mnt/target/  

Flags:

• -r: Recursive (copies subdirectories).
• -v (optional): Verbose mode.

Post-Migration Verification

After migration, verify data integrity to ensure no files were corrupted or missed.

1. Check Size Matching

Compare the total size of source and target:

du -sh /mnt/source  
du -sh /mnt/target  

Sizes should be nearly identical (small differences may exist due to filesystem overhead).

2. Verify File Integrity

Use rsync with --checksum to compare file hashes (detects silent corruption):

sudo rsync -av --checksum --dry-run /mnt/source/ /mnt/target/  

A “dry run” (--dry-run) shows if files differ without making changes.

3. Check Permissions and Ownership

Ensure permissions match the source:

ls -l /mnt/source/some_file  
ls -l /mnt/target/some_file  

Fix mismatches with chmod or chown if needed.

4. Update System Configuration (if migrating OS drive)

If migrating a system drive (e.g., from HDD to SSD), update boot settings:

• Update /etc/fstab: Use blkid to get the target partition’s UUID and replace the source’s UUID in /etc/fstab:

  sudo blkid /dev/sdY1  # Get target UUID  
  sudo nano /etc/fstab  # Replace old UUID with new one  

• Update GRUB: Rebuild the bootloader to recognize the new drive:

  sudo update-grub  

Troubleshooting Common Issues

1. “Permission Denied” Errors

• Use sudo for migration commands (required for system files).
• Check if the source/target is mounted with ro (read-only). Remount as read-write:

  sudo mount -o remount,rw /mnt/source  

2. Insufficient Space on Target

• Check free space with df -h /mnt/target.
• Delete unnecessary files from the source or use a larger target device.

3. dd Taking Too Long

• Increase the block size (bs=) to 8M or 16M (e.g., bs=16M).
• Avoid interrupting dd—it cannot resume mid-transfer.

4. Target Not Mounting

• Check for filesystem errors with fsck:

  sudo fsck /dev/sdY1  

• Verify the partition path with lsblk.

Conclusion

Migrating data between Linux storage devices is straightforward with the right tools and preparation. Always start with a backup, verify device paths, and choose the migration method that fits your needs (e.g., rsync for flexibility, dd for clones). Post-migration verification ensures your data is intact, and updating system configs (for OS drives) prevents boot issues.

By following these steps, you can safely transfer data and upgrade your storage with confidence.

References

• rsync Man Page
• dd Man Page
• Parted Documentation
• Linux Storage Administration Guide
• Smartmontools (smartctl) Guide