Standard Operating Procedure For Fixing STAB Entry
Standard Operating Procedure For Fixing STAB Entry
SOP Lists steps for Fixing Incorrect /etc/fstab Entry via AWS SSM After Boot Failure (Maintenance Mode).
To recover a Linux-based EC2 instance that fails to boot or enters maintenance mode due to invalid entries in the /etc/fstab file, by using AWS Systems Manager (SSM) to access and correct the configuration.
Applies to Amazon EC2 Linux instances where an invalid entry in /etc/fstab causes system boot failure, login issues, or sends the instance into emergency/maintenance mode.
Instance becomes unreachable via SSH.
Instance console output shows:
Welcome to emergency mode! After logging in, type "journalctl -xb" to view system logs...
Errors like:
Cannot mount /mnt/data - special device does not exist
Failed to mount /mnt/data.
Dependency failed for Local File Systems.
Boot process halts with messages such as:
You are in emergency mode. Press Enter for maintenance or Ctrl+D to continue.
Misconfigured /etc/fstab file:
Invalid UUID or device path.
Missing nofail option on non-critical volumes.
Wrong mount options or syntax errors
Failed boot or changed to maintain ace mode of EC2 instance.
EC2 instance must be in running state.
Instance must have:
SSM Agent installed and running.
IAM role with AmazonSSMManagedInstanceCore policy attached.
Access to SSM endpoints (via public internet or VPC endpoint).
IAM user must have permissions to run SSM commands and start sessions
Recovery Procedure
Step 1: Access AWS Systems Manager
Login to the AWS Console.
Navigate to Systems Manager > Run Command.
Step 2: Use SSM Run Command to Fix /etc/fstab
Click Run Command.
Select the document AWS-RunShellScript.
Under Command Parameters, paste the following script:
# Backup the current fstab for safetycp /etc/fstab /etc/fstab.bak
# Comment out all lines except for root (/) mount
sed -i '/^[^#]/s/^/#/' /etc/fstab
# Optionally, print the file for logscat /etc/fstab
In Targets, select the affected EC2 instance.
Click Run.
Step 3: Start a Session and Validate
Go to Systems Manager > Session Manager.
Start a session with the instance.
Verify the file is now safe:
cat /etc/fstab
Only the root (/) mount entry should remain active; others should be commented out.
Step 4: Reboot the Instance
Now that the system will not attempt invalid mounts:
sudo reboot
Wait for the instance to reboot and verify access via SSH or Session Manager.
Step 5: Identify and Correct the Problem
Once inside the system:
Open /etc/fstab:
sudo vi /etc/fstab
Uncomment or correct the problematic lines.
Common corrections:
Use correct device path:
/dev/xvdf1 /mnt/data ext4 defaults,nofail 0 2
Add nofail option to non-essential mounts:
UUID=xxxx-xxxx /mnt/data ext4 defaults,nofail 0 2
Save and exit the editor.
Test the configuration:
sudo mount -a
Check for errors. If none appear, the file is valid.
Optional: Restore from Backup
If needed, restore the original file:
cp /etc/fstab.bak /etc/fstab
Post-Recovery Cleanup
Validate mounts persist after reboot:
sudo reboot
Remove backup if no longer needed:
rm /etc/fstab.bak
Preventive Measures
Always use mount -a to validate changes before reboot.
Use nofail or failok options for non-critical mounts.
Monitor system logs after changes with:
journalctl -xb
EC2 instance up and running without any faults.