Craig Ringer <craig AT postnewspapers.com DOT au> writes:
> ----------------------------------------
> [A] AFAIK all it _needs_ for restore is:
> ----------------------------------------
>
> - File name
> - [st_size] File size in bytes (for restore verification)
> - md5sum
neither md5sum nor filename are stored in lstat, but that's just a
quibble.
> -----------------------------------------------------------
> [B] Things to keep to help users identify files to restore:
> -----------------------------------------------------------
>
> - [st_uid] User ID (not user name)
> - [st_gid] Group ID (not group name)
> - [st_mtime] Last-modified time
>
> ----------------------------------------
> [C] Things we can and should get rid of:
> ----------------------------------------
>
> - [st_dev] ID of device containing file
> - [st_ino] Inode number
these are needed in the future to handle hardlinks more optimally.
to illustrate the problem: I have a server with lots of content where
read access is granted by making hardlinks in each user's personal
directory. unfortunately, making a hardlink updates ctime, so this
action potentially causes thousands(!) of copies of the same file to
be included in an incremental backup. luckily, Bacula will only store
the actual data once, but it's still an annoyance, and in fact I'd
ideally not store the data again at all. (hard to implement in
today's catalog model, though.) I can make Bacula use mtime instead
of ctime for this job, but then new hardlinks will be ignored
completely in the incremental backup.
> - [st_ctime] Time of last status/inode change
this is very useful for forensic purposes.
> - [st_atime] Last accessed time
> - [st_rdev] Device ID (if special file)
> - [st_blksize] System I/O block size
> - [st_blocks] Number of 512B (not "system block size"?) blocks
I don't have any use for these.
> --------------------------
> [D] Ones I'm uncertain of:
> --------------------------
>
> - [st_nlink] Number of hardlinks
>
> ... which is useless, except if you want to try to restore
> hardlinks, in which case it might be a handy check value. On one
> hand, it's in the volume file. On the other hand, if we do something
> that'll make database designers cry and store NULL for values of 1,
> it takes up practically no space.
I'm not sure about this, either. you can make a relatively accurate
estimate for how much space a restore will require by just dividing
size by nlinks. to get a completely accurate number, you can analyse
the dev/ino tuples, but in a job with millions of files, that can be
awkward and costly to do in SQL.
> - [st_mode] Access mode (file system permissions)
>
> It's already in the volumes, so it's availible for proper restoration.
>
> Dropping mode does mean you can't verify restored mode against the
> mode in the catalog, so mode *might* want to be kept
> anyway. Personally I don't think it's worth having, since it doesn't
> add much when selecting files.
one use case is to allow ordinary users to restore all files they can
read, not only the files they own (in a future enhancement). not too
compelling, really.
> --------------------------------------
> [E] Handy extras not currently tracked
> --------------------------------------
>
> - Owning user _name_
> - Owning group _name_
>
> These can be tracked without storing them individually per-file, by
> maintaining a per-job uid->user, gid->group mapping in a separate
> "idmap" table. I'm not sure it's worth it, when it's so easy to just
> look up the uid/gid on the host you're interested in, but maybe it's
> worthwhile. Anybody?
hrm, this table can quickly get very large for some clients, since you
need to store the user database for each *job*. if you want to do
this, I think it's better to use a "Filename" style table. "kjetilho"
is just one row and one OwnernameId, reused across all clients and
jobs. (there is no guarantee that it's the same entity.)
> - Owning user SID
> - File creation time
no opinion since I only have Unix hosts.
> In the above, mtime is defined as a database `timestamp' rather than
> stored as the raw integer value from the source host. Note that the
> database may not store timestamps with the same precision as the fd
> host, and may store them using an internal floating-point form, so
> some error must be allowed for during verification.
ok.
> The alternative is storing a `bigint' (64-bit integer) which has a
> meaning depending on the timestamp precision and epoch of the origin
> system. Not, IMO, that useful.
agreed.
> The `checksum' (was: "md5") field MUST be NULL if if
> unset/unknown/unknowable (such as for a directory or device node),
> NOT zero. This saves space on many DBMSs, and is semantically
> cleaner.
sounds good to me.
> It'd be interesting to look at storing `checksum' as a bytea/blob
> field instead of base64-encoding it, but the gains probably wouldn't
> be too huge, converting DBs from bacula's wacky almost-base64 would
> be "interesting", and the FD sends base64-encoded data currently so
> there'd be compatibility issues to deal with. Not worth it.
I think it would be worth it, it's six bytes per row, or 5%.
migrating to the new format will take *ages* anyway, so I don't think
it is a good idea to require users to do it. better to first check in
File for data stored according to the new schema, then in FileOld (or
something) for data stored before the switch. the code to look in
FileOld can be killed some time later.
(I'm dreading the switch to 3.0... I seriously doubt I'll be able to
do it in 24 hours, and I don't like the thought of a day without
backups. but we'll see.)
--
Kjetil T. Homme
Redpill Linpro AS - Changing the game
------------------------------------------------------------------------------
Stay on top of everything new and different, both inside and
around Java (TM) technology - register by April 22, and save
$200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco.
300 plus technical and hands-on sessions. Register today.
Use priority code J9JMT32. http://p.sf.net/sfu/p
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
|
