Backups - Revision history

imported>Huston at 19:42, 24 August 2020

2020-08-24T19:42:05Z

← Older revision		Revision as of 19:42, 24 August 2020
Line 1:		Line 1:
	Data stored on hard drives has become very precious to many. In some cases, the ones and zeros written to disk can mean weeks, months or years worth of work to their author, and in other cases (such as data from a telescope run) they can never be recovered. For this reason, we not only have some redundant systems in place, but also some backup systems to try to make sure important data is never lost.		Data stored on hard drives has become very precious to many. In some cases, the ones and zeros written to disk can mean weeks, months or years worth of work to their author, and in other cases (such as data from a telescope run) they can never be recovered. For this reason, we not only have some redundant systems in place, but also some backup systems to try to make sure important data is never lost.
			{{update}}


Line 45:		Line 46:
	If you have a laptop and want to back it up (you should, unless you wouldn't care at all if all the data disappeared suddenly - and since laptops travel around, they're more prone to breakage and drive failure than most desktops), there's a couple options.		If you have a laptop and want to back it up (you should, unless you wouldn't care at all if all the data disappeared suddenly - and since laptops travel around, they're more prone to breakage and drive failure than most desktops), there's a couple options.

	~~=== Personal~~ TSM backups ~~===~~		{{update\|The information that was here on TSM backups is no longer valid; information about CrashPlan should be put here.}}
	~~Pros:~~
	* Backups are handled by the software automatically (with few exceptions)
	* Off-site storage decreases chance of laptop and backup both falling victim to a disaster
	* You don't have to manage the backup storage, TSM handles it for you
	~~Cons:~~
	* Your laptop must be ~~plugged in to the network to backup (wireless will work in a pinch, but be painfully slow)~~
	* If you need to restore a file, you must be plugged in to the campus network
	* If you write data while on a trip, and aren't back in the building before a failure, you won't have a backup
	~~See the TSM link above for information on registering your laptop for the TSM backup services~~.

imported>Huston: Backup changes due to OIT's file server being /u

2013-07-26T15:32:32Z

Backup changes due to OIT's file server being /u

← Older revision		Revision as of 15:32, 26 July 2013
Line 3:		Line 3:

	== What is backed up? ==		== What is backed up? ==
	Unfortunately, backing up data (or keeping it on highly reliable media) can be expensive. Because of that, we cannot afford to backup everything. However, everything that is stored in /u is backed up, as well as kept on a very reliable NFS server ~~(made~~ by ~~[http://www.netapp.com Network Appliance])~~. We also backup everything from the email server.		Unfortunately, backing up data (or keeping it on highly reliable media) can be expensive. Because of that, we cannot afford to backup everything. However, everything that is stored in /u is backed up, as well as kept on a very reliable NFS server maintained by OIT. We also backup everything from the email server.

	'''''We do not backup scratch disks!''''' The very nature of a scratch disk is that it's used for temporary storage of data and intermediate files; perhaps a local copy of data you have on CD elsewhere, or some outputs for in-between states of a data pipeline. Read [[Scratch disks]] for more information.		'''''We do not backup scratch disks!''''' The very nature of a scratch disk is that it's used for temporary storage of data and intermediate files; perhaps a local copy of data you have on CD elsewhere, or some outputs for in-between states of a data pipeline. Read [[Scratch disks]] for more information.
Line 9:		Line 9:
	== On-site backups ==		== On-site backups ==
	=== Snapshots ===		=== Snapshots ===
	The ~~central~~ file server (which stores your home directory in /u) has a feature available called snapshotting. Every two hours a "snapshot" of the current state of your files is taken and frozen. This is useful if you accidentally delete a file or discover that a file is corrupt, as it means you can "roll back the clock" so to speak and recover an earlier version of your file.		The OIT-maintained file server (which stores your home directory in /u) has a feature available called snapshotting. Every two hours a "snapshot" of the current state of your files is taken and frozen. This is useful if you accidentally delete a file or discover that a file is corrupt, as it means you can "roll back the clock" so to speak and recover an earlier version of your file.

	Snapshots for your directory are located in the ~/.snapshot directory (you won't see it in a directory list, but it is there if you '<tt>cd</tt>' to it). Under that directory you'll see other directories with names like "<tt>hourly.0</tt>" or "<tt>nightly.0</tt>". The higher the number, the older the snapshot is. <tt>hourly.0</tt> was taken at the last even hour, <tt>hourly.1</tt> was from two hours before that, and so on. The nightly snapshots are taken at midnight, so <tt>nightly.0</tt> is the state of your files at midnight this morning, and <tt>nightly.1</tt> is midnight the day before.		Snapshots for your directory are located in the ~/.snapshot directory (you won't see it in a directory list, but it is there if you '<tt>cd</tt>' to it). Under that directory you'll see other directories with names like "<tt>hourly.0</tt>" or "<tt>nightly.0</tt>". The higher the number, the older the snapshot is. <tt>hourly.0</tt> was taken at the last even hour, <tt>hourly.1</tt> was from two hours before that, and so on. The nightly snapshots are taken at midnight, so <tt>nightly.0</tt> is the state of your files at midnight this morning, and <tt>nightly.1</tt> is midnight the day before.

			{\| class="wikitable"
			\|+ Snapshot Schedule
			\|-
			! Name
			! Frequency
			! Number
			\|-
			\| Hourly
			\| Every 2 hours (even)
			! 12
			\|-
			\| Nightly
			\| Every night at midnight
			! 7
			\|-
			\| Weekly
			\| Every Monday at midnight
			! 4
			\|}


	==== How to recover via snapshots ====		==== How to recover via snapshots ====
	To recover a snapshot, just copy the file from the snapshot directory to anywhere in your home directory. Note that you won't be able to delete, rename, or otherwise modify anything under .snapshot as it's read only.		To recover a snapshot, just copy the file from the snapshot directory to anywhere in your home directory. Note that you won't be able to delete, rename, or otherwise modify anything under .snapshot as it's read only.

	If you need to recover a file from your home directory older than the available snapshots, please [[Requesting assistance\|contact us]] as soon as possible so we can recover it from the [[#Off-site backups\|nightly backups]] which are performed on the central file server.


	== Off-site backups ==		== Off-site backups ==
	~~Every night, starting around 0130,~~ the ~~home directory~~ server is backed up ~~to OIT's [http://www.princeton.edu/tsm TSM] servers. This doesn't always succeed - sometimes their servers are overloaded and~~ backups ~~don't complete, but that's why we have the [[#Snapshots\|snapshot]] feature turned on. Files that are backed up via TSM~~ are ~~kept~~ for ~~90 days from the date of their deletion,~~ and ~~the last four revisions of a~~ file ~~are kept as well~~. ~~This means that if~~ you ~~discover~~ a file is ~~corrupted~~, ~~and~~ it's ~~been changed~~ three ~~or less~~ times, ~~we can probably recover~~ a ~~version~~ of the file ~~before it was corrupted.~~		While the file server is backed up, those backups are for disaster recovery only and not file recovery. If you need a file restored, you must get it from the available snapshots - if the file is not there, it is not recoverable. OIT's backups are performed every two weeks and are rotated three times, thus making roughly 6 weeks of data available should there be a catastrophic failure of the file system.

	~~'''NOTE:'''~~
	: To give the best chance that we can recover a file for you, you '''''MUST''''' [[Requesting assistance\|contact us]] as soon as you realize you need a file recovered! The longer you wait, the less chance there is that we can get the file back.


	~~=== Recovering files via TSM ===~~
	Due to the way TSM operates, you cannot restore your own files that were on the home directory server. We will have to run the restore for you. Please [[Requesting assistance\|contact us]] ''as soon as you realize you need a file restored'' so we have a better chance of recovering it for you.

imported>Huston at 17:19, 21 May 2007

2007-05-21T17:19:10Z

← Older revision		Revision as of 17:19, 21 May 2007
Line 17:		Line 17:
	To recover a snapshot, just copy the file from the snapshot directory to anywhere in your home directory. Note that you won't be able to delete, rename, or otherwise modify anything under .snapshot as it's read only.		To recover a snapshot, just copy the file from the snapshot directory to anywhere in your home directory. Note that you won't be able to delete, rename, or otherwise modify anything under .snapshot as it's read only.

	If you need to recover a file from your home directory older than the available snapshots, please ~~send an e-mail to help@astro~~ as soon as possible so we can recover it from the [[#Off-site backups\|nightly backups]] which are performed on the central file server.		If you need to recover a file from your home directory older than the available snapshots, please [[Requesting assistance\|contact us]] as soon as possible so we can recover it from the [[#Off-site backups\|nightly backups]] which are performed on the central file server.


Line 24:		Line 24:

	'''NOTE:'''		'''NOTE:'''
	: To give the best chance that we can recover a file for you, you '''''MUST''''' ~~send email to help@astro~~ as soon as you realize you need a file recovered! The longer you wait, the less chance there is that we can get the file back.		: To give the best chance that we can recover a file for you, you '''''MUST''''' [[Requesting assistance\|contact us]] as soon as you realize you need a file recovered! The longer you wait, the less chance there is that we can get the file back.


	=== Recovering files via TSM ===		=== Recovering files via TSM ===
	Due to the way TSM operates, you cannot restore your own files that were on the home directory server. We will have to run the restore for you. Please ~~email help@astro~~ ''as soon as you realize you need a file restored'' so we have a better chance of recovering it for you.		Due to the way TSM operates, you cannot restore your own files that were on the home directory server. We will have to run the restore for you. Please [[Requesting assistance\|contact us]] ''as soon as you realize you need a file restored'' so we have a better chance of recovering it for you.

imported>Huston: Added section about backing up laptops

2007-05-09T16:53:38Z

Added section about backing up laptops

@@ Line 29: / Line 29: @@
 === Recovering files via TSM ===
 Due to the way TSM operates, you cannot restore your own files that were on the home directory server.  We will have to run the restore for you.  Please email help@astro ''as soon as you realize you need a file restored'' so we have a better chance of recovering it for you.
 == See also ==
 * [[Scratch disks]]

imported>Huston: Added blurbs about scratch disks & what is backed up

2007-05-04T21:10:14Z

Added blurbs about scratch disks & what is backed up

← Older revision		Revision as of 21:10, 4 May 2007
Line 1:		Line 1:
			Data stored on hard drives has become very precious to many. In some cases, the ones and zeros written to disk can mean weeks, months or years worth of work to their author, and in other cases (such as data from a telescope run) they can never be recovered. For this reason, we not only have some redundant systems in place, but also some backup systems to try to make sure important data is never lost.


	~~Data stored~~ on ~~hard drives has become very precious to many~~. ~~In some cases~~, ~~the ones and zeros written~~ to ~~disk can mean weeks~~, ~~months or years worth of work to their author~~, ~~and in other cases (such~~ as ~~data from~~ a ~~telescope run~~) ~~they can never be recovered~~. ~~For this reason, we not only have some redundant systems in place, but~~ also ~~some~~ backup ~~systems to try to make sure important data is never lost~~.		== What is backed up? ==
			Unfortunately, backing up data (or keeping it on highly reliable media) can be expensive. Because of that, we cannot afford to backup everything. However, everything that is stored in /u is backed up, as well as kept on a very reliable NFS server (made by [http://www.netapp.com Network Appliance]). We also backup everything from the email server.

			'''''We do not backup scratch disks!''''' The very nature of a scratch disk is that it's used for temporary storage of data and intermediate files; perhaps a local copy of data you have on CD elsewhere, or some outputs for in-between states of a data pipeline. Read [[Scratch disks]] for more information.

	== On-site backups ==		== On-site backups ==
Line 25:		Line 29:
	=== Recovering files via TSM ===		=== Recovering files via TSM ===
	Due to the way TSM operates, you cannot restore your own files that were on the home directory server. We will have to run the restore for you. Please email help@astro ''as soon as you realize you need a file restored'' so we have a better chance of recovering it for you.		Due to the way TSM operates, you cannot restore your own files that were on the home directory server. We will have to run the restore for you. Please email help@astro ''as soon as you realize you need a file restored'' so we have a better chance of recovering it for you.

			== See also ==
			* [[Scratch disks]]

imported>Huston: Merged a few topics into this one

2007-05-04T20:58:39Z

Merged a few topics into this one

@@ Line 1: / Line 1: @@
-{{oldfaq|NUM=7}}
 Data stored on hard drives has become very precious to many.  In some cases, the ones and zeros written to disk can mean weeks, months or years worth of work to their author, and in other cases (such as data from a telescope run) they can never be recovered.  For this reason, we not only have some redundant systems in place, but also some backup systems to try to make sure important data is never lost.
-Answers in this category:
-*[[Backup Frequency]] or "How often are files backed up?"
+== On-site backups ==
-*[[Backup Revisions]] or "How many revisions are kept?"
+=== Snapshots ===
-*[[Restoring Files]]
+The central file server (which stores your home directory in /u) has a feature available called snapshotting.  Every two hours a "snapshot" of the current state of your files is taken and frozen.  This is useful if you accidentally delete a file or discover that a file is corrupt, as it means you can "roll back the clock" so to speak and recover an earlier version of your file.

imported>Huston at 20:44, 4 May 2007

2007-05-04T20:44:23Z

← Older revision		Revision as of 20:44, 4 May 2007
Line 1:		Line 1:
	{{oldfaq\|NUM=7}}		{{oldfaq\|NUM=7}}
	~~Questions about departmental~~ and ~~campus-wide~~ backup systems		[[User:Huston\|huston]] 16:44, 4 May 2007 (EDT)
			{{inuse}}
	~~----~~		Data stored on hard drives has become very precious to many. In some cases, the ones and zeros written to disk can mean weeks, months or years worth of work to their author, and in other cases (such as data from a telescope run) they can never be recovered. For this reason, we not only have some redundant systems in place, but also some backup systems to try to make sure important data is never lost.

	Answers in this category:		Answers in this category:

imported>Lkoven: New page: {{oldfaq|NUM=7}} Questions about departmental and campus-wide backup systems ---- Answers in this category: Backup Frequency or "How often are files backed up?" [[Backup Revisions]...

2007-05-02T18:35:02Z

New page: {{oldfaq|NUM=7}} Questions about departmental and campus-wide backup systems ---- Answers in this category: *Backup Frequency or "How often are files backed up?" *[[Backup Revisions]...

New page

{{oldfaq|NUM=7}}
Questions about departmental and campus-wide backup systems

----

Answers in this category:
*[[Backup Frequency]] or "How often are files backed up?"
*[[Backup Revisions]] or "How many revisions are kept?"
*[[Restoring Files]]