DAM organization - getting there but a few ??

[iamback]

I'm finally making real progress setting up my data structures for my DAM (Digital Assets Management). I had to, since I came back with a ton of digital images from my last trip! :)

The problem was that the literature generally assumes all your files (images) come from a single source (say, a scanner, or a digital camera) but I need to combine files from a range of different sources into one data structure - and then to manage it all with a single database.

For example, I have negative film (images to be scanned, some scanned with an old film scanner), contributed and inherited slides (to be scanned), scanned prints (of my own photographs - to be substituted later by scans from negatives); contributed images provided as digital camera files, negatives (to be scanned), prints (scanned or to be scanned) and downloaded (public domain or used with permission). I've used two flatbed scanners, and two film scanners (one of each current but I have files from all). I also have digital files from cameras, two camphones (one current) and now also the Fuji Finepix F30 compact camera. And then are the files I myself create on the computer, directly, or via screen capture. That's the summary. ;-) You get the idea.

The "DAM Book" (can't find the thread where I mentioned it - "Search" seems to be essentially non-functional now) suggests using "buckets" to archive images in - folders that grow to a size that just fits on a CD or DVD and then get archived to that media. I like this idea, but it is simple only when it all comes from a single source and you can work sequentially - but I can't and still wanted to use this useful concept.

So, I finally hit on a system that combines my "origins" (where the files come from) with the "buckets" idea for archiving; I've also devised a file naming system that supports this (with prefixes also indicating "origin" but otherwise in principle keeping original file names). Mostly, it works, and I've been busy moving a lot of stuff that I already had (and had already catalogued in IMatch (http://www.photools.com/) - the DAM database program I'm using). But now I'm hitting a few snags that I'd like your input on.

Basically, my directory structure now looks like this (see attachment): a DAM directory, with three main directories below that:
RAW - contains files as they get into the computer (not necessarily "RAW" format, but the very first digital form I get my hands on)
Master - contains files derived from RAW files that are used for as the basis further processing rather than the Raw format (for instance, the rotated form of portrait-format images, or a straightened or perspective-corrected image)
Derived - all further processed forms based on either Raw or Master files

Only RAW gets further subdivided into "origin" categories - see attached image:
Scan - for scanned images
Digital - for files from digital cameras
Ext - for files that are contributed or downloaded; one directory per copyright owner (initials or domain, plus 'NN' for unknown) and again further subdivided: for instance under koningaap.nl I have both downloaded and scanned route maps that I use on my travel blog (with permission)


Each of these nodes will then get its own series of "buckets", named RBnnnnnn, MBnnnnnn and DBnnnnnn (raw/master/derived) where 'nnnnnn' is a serial number within each of the groups (that means the names are all unique and can directly be used as volume names for CDs/DVDs).

Now for the two snags - two things that are not very well integrated yet, or I can't decide how to integrate, that I'd like your input on:

1. So far I have only a few files archived that I created myself (but I have tons to be integrated); I've called the directory "Draw" but that is not a very good name. There would be logos, drawings, screenshots, icons, and similar. Suggestions for a better name? And this should probably also be subdivided into directories before getting to "buckets" - by creation method? (drawing, screenshot, icon...) Something else (image format)?

2. I also encountered a file (and I'll have many more of those) that I created but dows not usefully fit into the class above: I found that specific cameras or scanners may have particular image characteristics that always need the same corrections. For instance an adjustment layer, or set of adjustment layers to remove a color cast; or a preset to correct distortion of a particular lens; etc. Rather than classify these by origin as "created", I think it would be more useful to classify them by target: files of a particular origin that they may be applied to. So they don't logically fit into the RAW/Draw branch - but where to put them? A fourth directory below DAM (with further subdivisions maybe)?

All suggestions and comments welcome!

[ktinkel]

Have nothing cogent to say about the process, but can link you to two old threads here:

Principles of photo management
http://desktoppublishingforum.com/bb/showthread.php?t=1811

Ad tracking/DAM software?
http://desktoppublishingforum.com/bb/showthread.php?t=856

And I do share your concern about our search here; I found both of these via Google. Looking into the problem, but at least you can read the old discussions.

[Bo Aakerstrom]

Just a few thoughts really:

I have negative film (images to be scanned, some scanned with an old film scanner), contributed and inherited slides (to be scanned), scanned prints (of my own photographs - to be substituted later by scans from negatives); contributed images provided as digital camera files, negatives (to be scanned), prints (scanned or to be scanned) and downloaded (public domain or used with permission). I've used two flatbed scanners, and two film scanners (one of each current but I have files from all). I also have digital files from cameras, two camphones (one current) and now also the Fuji Finepix F30 compact camera. And then are the files I myself create on the computer, directly, or via screen capture. That's the summary. ;-) You get the idea.
I do, but I can't help but feeling that you are overcomplicating this!

The "DAM Book" (can't find the thread where I mentioned it - "Search" seems to be essentially non-functional now) suggests using "buckets" to archive images in - folders that grow to a size that just fits on a CD or DVD and then get archived to that media. I like this idea, but it is simple only when it all comes from a single source and you can work sequentially - but I can't and still wanted to use this useful concept.
The rest of us calls these "buckets" folders, but as a conceptual approach I suppose it works.:)

What happens if one of your buckets fills up quickly, whilst another one takes a loooong time to reach archiving size? The resulting CDs/DVDs are going to have content from varying time spans - would that not make it kind of hard to locate specific files later on?

Will you in addition to the have some kind of record of what's where?

Basically, my directory structure now looks like this (see attachment): a DAM directory, with three main directories below that:
RAW - contains files as they get into the computer (not necessarily "RAW" format, but the very first digital form I get my hands on)
Master - contains files derived from RAW files that are used for as the basis further processing rather than the Raw format (for instance, the rotated form of portrait-format images, or a straightened or perspective-corrected image)
Derived - all further processed forms based on either Raw or Master files

Only RAW gets further subdivided into "origin" categories - see attached image:
Scan - for scanned images
Digital - for files from digital cameras
Ext - for files that are contributed or downloaded; one directory per copyright owner (initials or domain, plus 'NN' for unknown) and again further subdivided: for instance under koningaap.nl I have both downloaded and scanned route maps that I use on my travel blog (with permission)

Using RAW as a name for unprocessed files is a bit ambigous, you might want to reserve that name for RAW image files later.

Each of these nodes will then get its own series of "buckets", named RBnnnnnn, MBnnnnnn and DBnnnnnn (raw/master/derived) where 'nnnnnn' is a serial number within each of the groups (that means the names are all unique and can directly be used as volume names for CDs/DVDs).
That makes sense to me.

I've called the directory "Draw" but that is not a very good name. There would be logos, drawings, screenshots, icons, and similar. Suggestions for a better name?\
Perhaps "images" (as opposed to scan, digital or ext) would work here. Finding names for folders in a system like this is always hard - a bit like some work I did for a market research company in a distant past (it was large spreadsheets, but the implications are the same). Consider what your future requirements might be.

And this should probably also be subdivided into directories before getting to "buckets" - by creation method? (drawing, screenshot, icon...) Something else (image format)?
You have to decide what is important here; how the content was originated or what you need to do with it after it is in the system.

2. I also encountered a file (and I'll have many more of those) that I created but dows not usefully fit into the class above: I found that specific cameras or scanners may have particular image characteristics that always need the same corrections. For instance an adjustment layer, or set of adjustment layers to remove a color cast; or a preset to correct distortion of a particular lens; etc. Rather than classify these by origin as "created", I think it would be more useful to classify them by target: files of a particular origin that they may be applied to. So they don't logically fit into the RAW/Draw branch - but where to put them? A fourth directory below DAM (with further subdivisions maybe)?
The danger here is that you'll end up with multiple copies of images/photographs in various stages of archiving/completion/formats - you could easily loose track of it all.

Not any real solutions, see it a online brainstorming!

[terrie]

bo: I do, but I can't help but feeling that you are overcomplicating this!I think so too but then, it's not my data...

My approach would probably be to take advantage of keywording to indentify the source of the image--camera type, scanner, date short, image category, shot location, etc.--but then, that's me and my data and what works for me may not work for someone else...

I use the term "original" (usually a suffix of "orig") for either the original camera image--be it digicam or scan of either a neg/slide or print--and then "work" or "wrk" for the image in process (might be pswrk or pntwrk for Photoshop working image, Painter working image) and then "fin" for the final--a lot depends on the particular image what "final" means and there may be multiple finals based on size--for example, a suffix of "5x7fin", "8x10fin", etc.

Probably as many ways to go about organizing as there are people...I figure it probably only really matters that one can find the image one is looking for without too much difficulty...'-}}

Terrie

[Bo Aakerstrom]

For me it would be overmuch, but as long as it makes sense to Marjolein it's working!

I have a collection of transparancies, negatives and prints going back to 1974 and with nothing more than a label on each container (with a short description of the contents and a date/catalogue number) together with keeping things stored in order, I can lay my hands on pretty much anything.

Admittedly, it is easier when it is something physical rather than a file on a harddrive or a CD/DVD ROM.

[terrie]

bo: it is easier when it is something physical rather than a file on a harddrive or a CD/DVD ROM.Indeed...holding a CD up to the light doesn't get you much...'-}}

Terrie

[Bo Aakerstrom]

I haven't tried that one!
Remember reading about that man who could tell you what was recorded on an LP by just looking at the grooves, but that wouldn't work with CDs either...

[iamback]

Thanks, all - some good feedback here that allows me to "get out of the rut" I had been thinking myself into. This stuff is really hard to get right!

I have negative film (images to be scanned, some scanned with an old film scanner), contributed and inherited slides (to be scanned), (...) That's the summary. ;-) You get the idea.I do, but I can't help but feeling that you are overcomplicating this!Well, yes and no. :) The point is that all the literature I've seen so far assumes you have images from only one or two sources and can essentially archive them sequentially. But I don't and I can't - it's my reality that is a whole lot more complicated!

It is in this more-complicated reality that I need to be able to find back, say, all "market" pictures, or all "townscapes". Such classification is the job of the database program (for which I have IMatch) but the images need to stored somewhere first, and (preferably) all have unique names as well. There are three sides to all this, storage, archiving and cataloguing; the solution I designed supports all three including my complicated reality:
Storage - that's what the directory structure and my file naming convention is for, which reflects my "complicated reality";
Archiving - that's what the "bucket" system is for, integrated into my
storage system; and
Cataloguing - which is where IMatch takes over (with the category trees I've designed and am still extending, combined with other meta data like EXIF, IPTC and special database fields.It's like having three dimensions: origin and ownership, archival location, and classification.

The rest of us calls these "buckets" folders, but as a conceptual approach I suppose it works.:) They are folders, and I mentioned that - but folders with a special set of properties, one of which is that they have a size limit (to correspond with the ultimate archival media). "Bucket" is a good name, because it suggests that fixed size: you fill one bucket and then move on to the next one.

What happens if one of your buckets fills up quickly, whilst another one takes a loooong time to reach archiving size? The resulting CDs/DVDs are going to have content from varying time spans - would that not make it kind of hard to locate specific files later on?This is one of the things that made it hard at first to combine the "buckets" idea (originally simply sequential) with the fact that with my files from different origins I can't have such a simple sequence. (This was already true once I started working with multiple analog cameras at the same time - I had to redesign my (physical) film numbering system to take that into account, and over the years it took three tries to get a consistent system. I went through a large exercise earlier this year to physically renumber and re-archive all my analog material.)

My solution is the integration of the buckets with my hierarchical directory structure, combined with IMatch to keep track of it all (the program doesn't care where you store a folder, so if I burn a bucket-ful to a CD, I can very simply tell the program to "relocate" those files to that CD, and it will keep track of it by its volume name). The database will still work whether the files are "on-line" or "off-line", but when I want to work with physical files that are on a particular CD, the program will prompt me for it. This is the major reason why you need a management program (with a database) rather than only a browser program, so you work with your material and classifications, regardless whether the files are on-line or off-line.

Using RAW as a name for unprocessed files is a bit ambigous, you might want to reserve that name for RAW image files later.I agree, and was a bit uneasy with the name, but I needed something to get really started. I considered "Original" but that doesn't work for me either for in the case of analog images I consider those the actual "originals". Maybe something that indicates "digital original" would work better ("OrigDig"?). As I indicated above, renaming a directory is no big deal with IMatch - I just use Explorer to rename, and then tell IMatch to "relocate" to the new location. Oh, and it needs to start with a different letter than "D" or "M" to keep my bucket-naming system consistent. "O" would work...

I've called the directory "Draw" but that is not a very good name. There would be logos, drawings, screenshots, icons, and similar. Suggestions for a better name?Perhaps "images" (as opposed to scan, digital or ext) would work here. Finding names for folders in a system like this is always hard - a bit like some work I did for a market research company in a distant past (it was large spreadsheets, but the implications are the same). Consider what your future requirements might be.Well, nearly all files are "images" in the first place (though I can integrate videos and even sound files) so that would be rather ambiguous. I just hit on "Computer" - as in produced with a computer rather than with a camera. (I consider a scanner as a type of camera.) That would probably work but I'm still open to a better name...

My approach would probably be to take advantage of keywording to indentify the source of the image--camera type, scanner, date short, image category, shot location, etc.--but then, that's me and my data and what works for me may not work for someone else...Indeed, those are some of the things that I let IMatch take care of, in a combination of my category system and special database fields. But that is separate from physical storage and file naming.

I use the term "original" (usually a suffix of "orig") for either the original camera image--be it digicam or scan of either a neg/slide or print--and then "work" or "wrk" for the image in process (might be pswrk or pntwrk for Photoshop working image, Painter working image) and then "fin" for the final--a lot depends on the particular image what "final" means and there may be multiple finals based on size--for example, a suffix of "5x7fin", "8x10fin", etc.My file prefixes indicate originating (digital) device and state (Original/Master/Derived); for variants I use suffixes where for "publication-ready" (and sometimes intermediates for a particular size) I also indicate a size (more or less) as the final part of the suffix. In principle I use the "largest" size in pixels, so I can have _h800, _w400 or _x150 (so 'h' is a portrait format, 'w' is a landscape format and 'x' indicates it's a square-sized image). In principle I can use these final suffixes in web scripts to automatically select and present appropriate versions of an image, because the naming is consistent (it becomes just a simple mask on the file name for the program to select a particular version).

But I very much like your idea of marking images as "final" - I hadn't thought of that yet! I'll probably encode that in the database (as a new subcategory under my "state" category). But I'm holding your thought - I may end up encoding state "final" (next to "Original/Master/Derived") in the file prefix as well. Because it's combined with suffixes, I also could have any number of "finals" that way and still have the files all "connected" by their names.

[iamback]

Workflow

The danger here is that you'll end up with multiple copies of images/photographs in various stages of archiving/completion/formats - you could easily loose track of it all.The trick is to use the management program (IMatch) as a central hub in any workflow.

For instance, when on the road on my last trip, each evening I would store all digital images taken on my image tank and then re-format the memory cards. The camera takes care of its own file numbering, the image tank adds a prefix and stores in directories that are named and numbered according to the type of memory card it's archiving. That worked flawlessly. Now that I'm back, I just plug the image tank into one of my computers and it becomes an external hard drive. The rest of the work flow then goes as follows:
Copy one directory off the image tank to a folder on my local HD
Let IMatch "scan" this folder - I can tell it to assign basic (common) categories during the scanning process; I use "state" ("original"), "origin" (camera name) and a basic "location". The scanning process also creates thumbnails in the database, and an "off-line cache" images.
Refine categorization; easy since one folder represents one day (two at most), so with my notes for the day I can easily assign a more precise location and subject categories for batches of images
Rename files with my extra prefix (if I already have master or derivative files, which I do in some cases this is where I take care of those)
Move or relocate the files to their "bucket" on the external storage drive

On my local HD I have a directory structure similar to that on the external media HD (minus buckets), that I use for this "staging" process, since it is a lot faster for IMatch to process the files from a local HD. When I'm done with one folder on the image tank, I end up with an empty folder on my local HD, and just repeat the process for the next folder on the image tank.

Since after this images are already categorized by location (and have a file date) it's easy to later select a group and refine or add extra categories - this will be purely database work in IMatch, the files don't have to be online for that (as long as the local cache and/or thumbnails have enough information for me to decide on the categorization).

My workflow to get images already stored into the same re-designed storage system (sometimes keeping already-assigned categorization) was just a variant of the above; and that's done for some large batches of already-existing digital files now.

Later, I can use IMatch to select files by their categories (or date, or image contents), make sure they're online, and from there load them into an image editor for making further derivatives; store those (with derived filename) locally, copy categorization (or change "original" to "derived"), and them move them to a bucket. And here it does not matter any more whether those buckets are filled sequentially - IMatch just keeps the database and can select by its metadata, and will know where the files reside physically. As long as I use IMatch as my "launch pad" to work with the files once they are catalogued - that's why any work flow will need to include that.

For scanning film, the whole workflow will be very similar: scan to a "staging" directory (using my new film/negative numbering as part of the file names) and once they're there, the basic "scanning" and cataloguing within IMatch will be the same as for images I pull off my image tank.

I'm also going to scan "objects" I bring from a trip as illustrations (things like tickets, but also small 3D objects); basically that process will also be the same, except that I haven't yet decided on a good file naming system for those files. I'll think of something... But again, once I have the digital files on my computer, the initial workflow to get them into IMatch will be essentially the same, and after that it will continue to be my launch pad for further working with the images. The only difference being that the actual originals are not negatives or slides, but physical objects.

Finally, part of my workflow is that each evening after a "batch" of work I make a (separate) backup of the IMatch database, which also goes onto my external (network) media drive. Drives, since I have two identical drives and mirror one to the other overnight; I will completely remove original files only once I have two copies on the network drives. And these drives are big, so I don't have to worry about archiving to CD/DVD any time soon. I can just dump files into the buckets for now and take care of archiving later.

Having a consistent naming system, keeping original file names essentially intact but with variations in prefix and suffix to indicate their "state" also enables me to always find back the original file even without the database. The only problem here is files that are already renamed for publication on the web or emailing them to people; for those, I'll have to indicate the actual corresponding file in the database somehow - I may create a special database field for that. (But I'll no longer rename files that way, other than with my prefix-and-suffix system which keeps them together.)


Open questions

So.. what's left?
"RAW" does need to be renamed - "OrigDigital" or something?
"Draw" needs to be renamed as well - "Images" is too ambiguous but maybe "Computer" as indicating the "origin" device of the images. I also need some file naming system where a computer program doesn't already do that automatically (such as with screenshots).
I still need a place to store my "adjustment" files with layers and presets - "Target" (with subdirectories for devices, and a general one) next to "OrigDigital", "Ext" and "Scan"? File naming could include a code for the target device (if any), just like I now use prefixes that encode "origin" device of the digital files.
A file naming system needed for scans of physical objects. Since film scans use my film number which encodes the year I "started" a film, the year I acquired the objects would probably be part of that but I haven't thought beyond that (except "subject" should not be part of it - subjects are what the database takes care of). I'm open to suggestions here.


All input and comments appreciated!

[iamback]

Thanks, Kathleen! At least that links the threads together now...

I suspect the search was somehow "killed" with upgrading the software. Maybe you need to do a "reindexing" run or something to make it all searchable again? Just browsing threads doesn't work if you don't know where to look for something in the first place, and especially not for older threads - that's what the search is intended for. :(

[ElyseC]

I went exploring in Visual Thesaurus (http://www.visualthesaurus.com/) (I bought the standalone version a while back) and here are some suggestions.
So.. what's left?
"RAW" does need to be renamed - "OrigDigital" or something?Perhaps Master?

"Draw" needs to be renamed as well - "Images" is too ambiguous but maybe "Computer" as indicating the "origin" device of the images. I also need some file naming system where a computer program doesn't already do that automatically (such as with screenshots).Right now the only terms that spring to mind are ambigous or long but maybe they'll spark a better idea:
manual, home-grown, hand-made, crafted

I still need a place to store my "adjustment" files with layers and presets - "Target" (with subdirectories for devices, and a general one) next to "OrigDigital", "Ext" and "Scan"? File naming could include a code for the target device (if any), just like I now use prefixes that encode "origin" device of the digital files.Could these be considered in progress, in process, processing or working images? Not sure what you mean by "target device" but naming code might be something like IPW-xxx for "in process/progress" files destined for web use.

A file naming system needed for scans of physical objects. Since film scans use my film number which encodes the year I "started" a film, the year I acquired the objects would probably be part of that but I haven't thought beyond that (except "subject" should not be part of it - subjects are what the database takes care of). I'm open to suggestions here.I use origin dates in file names frequently and it always requires I decide which is more important to sort by, date or subject. If I put the date code first, then in the OS's filing system it'll forever name-sort by those numbers.

[Cristen Gillespie]

Indeed...holding a CD up to the light doesn't get you much...'-}}

Terrie

LOL. That's why I print jewel case inserts. It's still not enough to know exactly where any given image is. But at least it eliminates looking at pictures of Alaska when I wanted to find a picture of Nova Scotia or my niece. ;-) Well, maybe ...

A decade or more into this and I'm pretty certain the only way I'll ever be able to lay my hands fairly quickly on a particular image I can remember existing is to have it catalogued in a program such as Portfolio (those thumbnails do help), and/or to have plenty of keywords/metadata (that I've created) to narrow the search. I might care about the date, or the origin, and can include that in the name or directory path, but I probably care even more about the content. Location helps, but some topics can be too large to find a single image easily, even if I've kept it the topic fairly well isolated from other topics.

There is also the problem of cross-referencing. It might be Alaska, but it might also be sunset or ocean or flora or fauna or architecture. Following? I want a picture of a train station in Alaska that was taken on a day when the sky was full of beautful clouds. How do I get there quickly if I have several CDs full of Alaska? Or maybe I'm not looking for Alaska, but clouds? How do I find my choices among the many cloud pictures if I've sorted, logically, by input, date, place, but not genre?

Where you add image progress to the filename (and perhaps keep the files all in one project folder -- an excellent solution and one used by many), I keep separate folders, naming the folders something that links originals to in progress to final. The folders themselves contain a series of images, such as "Michelle's Bachlorette Party" with the same name plus "Final" in a sub folder or a different folder at the same hierarchy level. The directory structure doesn't really matter to me because it will be eliminated once the project is complete.

But no matter how logical I attempt to be with folders, those folders alone won't help a search in the future if my need has changed from finding a picture I know was taken at a specific time and plact to finding a selection of images that spans time and place, but I still need it. And with as many pictures as I have, finding the best picture of Anasazi rock paintings among those taken over a 50 year span from various members of the family using various different cameras isn't going to happen without some serious keywording, or an awful lot of time spent looking at pictures of the West.

There are simply too many different needs to know exactly what advice to give that would have any meaning for another person. As you said, as many ways to get there as people.

[iamback]

I went exploring in Visual Thesaurus (http://www.visualthesaurus.com/) (I bought the standalone version a while back) and here are some suggestions.I had a look - interesting stuff...
Perhaps Master?No -that's not quote the same and I bewsides already use that term. I distinguish between these concepts:
Original - the actual first version of an image; with a digital camera, that's what the camera records (whether in a "RAW" format of something else like JPG or TIFF) but with analog images, it's the slide or negative (if I own that), or even a print (if that's the form in which an image is contributed)
What I now call "digital original" - the first form in which the file enters my computer system as a digital file; for files from a digital camera, that's (binary) identical to the original (although embedded meta information may be updated or added - as long as the imgae itself remains unchanged)
To make an image "usable" it may need basic "clean up" - things like noise reduction, removal of scratches and dust specks, straightening and perspective collection, or correction of a color cast. All those operations together (if needed) result in a master copy from which all other derivatives are made, instead of from the digital original.So there's an important difference between a (digital) original and a "master", which is why I actually treat and store them separately.
(The Visual thesaurus sort of hints at this but does not quite separate their meanings. But I'm not making up this usage of "master" - it's the version you make copies from, not an original recording, and used in this sense for both visual and sound material.)

"Draw" needs to be renamed as well - "Images" is too ambiguous but maybe "Computer" as indicating the "origin" device of the images.Right now the only terms that spring to mind are ambigous or long but maybe they'll spark a better idea:
manual, home-grown, hand-made, craftedOf those "home-grown" comes closest (but I'm not dropping "Computer" yet). The other terms suggest a manual process that may not be quite manual as in the case of a screen shot; but of course there are files that I created from scratch as well - so whatever name I use should include both (and exclude being made with a camera or scanner). Another way of looking at it is that no "external device" was used in their creation except the computer I'm working on.

I still need a place to store my "adjustment" files with layers and presets - "Target" (with subdirectories for devices, and a general one) next to "OrigDigital", "Ext" and "Scan"?Could these be considered in progress, in process, processing or working images? Not sure what you mean by "target device" but naming code might be something like IPW-xxx for "in process/progress" files destined for web use.No, "in progress, in process" refers more to the images themselves. These are more like "tools". For instance, a particular digital camera may always have a particular lens distortion - a preset to correct that (used in the process of creating a master file from the original!) could be stored to be reused to produce "masters" from all "original" files made with that particular camera. Or a particular scanner may always produce a particular color cast, and I may develop a curves adjustment layer to correct that: that adjustment layer may be stored as a file to be reused with all scans made with that particular scanner. So these files are not images "in progress" but files used for images from a particular device. Hence the idea of "target" device rather than "origin" to store them, but subdivided with the same list of external devices I already have (digital cameras and scanners). Naming the base directory to store them in "Target" doesn't quite work for me though - maybe something like "Tools" or "Resources" would be more meaningful.

A file naming system needed for scans of physical objects. Since film scans use my film number which encodes the year I "started" a film, the year I acquired the objects would probably be part of that (...)I use origin dates in file names frequently and it always requires I decide which is more important to sort by, date or subject. If I put the date code first, then in the OS's filing system it'll forever name-sort by those numbers.Well, I don't use anything to do with "subject" in file names any more (the database can keep track of subjects); for objects, a date would be "meaningful" though (at least for "travel illustrations") as that would at least tie the objects to a particular trip (just like my film numbers do). I could use ISO encoding with at least the year, extended with month, and day, where known. (For older stuff that I have kept but want to scan the exact date may not be known any more, but the year at least is known or can be reasonably guessed.) Beyond that date information I guess I could just use a numbering system and leave all cataloguing to be stored in the database itself. I'd end up with a naming scheme something like <prefix>_<O/M/D>_<year<-month<-day>>>_nnn_<suffix>.EXT (where prefix, as with other files, encodes the device used to create the digital files - in this case, the scanner, and the suffix is for keeping track of any derivatives).

Thanks for the great input!

Meanwhile, I have done some renaming of directories ("RAW" is now "OrigDitital", "Draw" is now "Computer" - but could be further changed) and I've used more complete camera names. Easy to make further changes, as I just rename in Explorer, and then tell IMatch (which notices the files are no longer "on-line") to "relocate" them to the new directory names; since this is merely a change in database data, it's fast and easy - and it won't allow you change to a non-existing directory anyway, so no errors are possible here.

My next step is to upgrade my copy of IMatch, before moving on to further tweaks to storage/naming and importing all my Korea/Beijing pictures!

[iamback]

A decade or more into this and I'm pretty certain the only way I'll ever be able to lay my hands fairly quickly on a particular image I can remember existing is to have it catalogued in a program such as Portfolio (those thumbnails do help), and/or to have plenty of keywords/metadata (that I've created) to narrow the search. I might care about the date, or the origin, and can include that in the name or directory path, but I probably care even more about the content. Location helps, but some topics can be too large to find a single image easily, even if I've kept it the topic fairly well isolated from other topics.

There is also the problem of cross-referencing. It might be Alaska, but it might also be sunset or ocean or flora or fauna or architecture. Following? I want a picture of a train station in Alaska that was taken on a day when the sky was full of beautful clouds. How do I get there quickly if I have several CDs full of Alaska? Or maybe I'm not looking for Alaska, but clouds? How do I find my choices among the many cloud pictures if I've sorted, logically, by input, date, place, but not genre?This is where a good cataloguing app comes into its own. In IMatch, I can do this easily with my subject categories (though you can also add keywords into IPTC or XMP data), and you can run "selections" on any number of criteria. The thing is to be as "complete" as possible in assigning keywords and/or categories.

For instance I have pictures taken at various markets around the world. My categories do not just take "market" and the location into account, but also (maybe) encode a particular ethnic group, and things "seen" on the image, like people, males or females, products or produce, clothing (are they wearing a hat, or a specific kind of hair or body decoration), activities (bargaining, waiting, making music) and so on. All of the things on the image that you may later want to find back. And when you see an image again and notice something that's not encoded yet, immediately add it.

I've spent considerable time setting up my categories system in IMatch, (using images from one trip as a starting point) and am now at the point where I can simply refine the categories (which include an extensive "tree of life" for taxonomy of all living creatures I take pictures of). (Internally, each category has an ID, so even if you move things around, anything that's already assigned to a particular category will stay assigned to it even if it's moved around.)

But no matter how logical I attempt to be with folders, those folders alone won't help a search in the future if my need has changed from finding a picture I know was taken at a specific time and plact to finding a selection of images that spans time and place, but I still need it. And with as many pictures as I have, finding the best picture of Anasazi rock paintings among those taken over a 50 year span from various members of the family using various different cameras isn't going to happen without some serious keywording, or an awful lot of time spent looking at pictures of the West.This is exactly why a good keywording/categories system in a cataloguing app is essential - and why you cannot encode subjects in directories or file names. Many images simply have a multitude of "subjects" or "topics" (rocks, trees, clouds, drawings, culture, location....) - you need a specialized database to keep track of all of those.

I had a quick look at Portfolio (and a bunch of other programs and reviews) when selecting an application but at least for the Windows platform could not find anything that even matches IMatch. It's very, very powerful, and so far I'm using only a fraction of its capabilities - but you can "grow into" it and not lose any of the meta data you've already stored.

[ElyseC]

Of those "home-grown" comes closest (but I'm not dropping "Computer" yet). The other terms suggest a manual process that may not be quite manual as in the case of a screen shot; but of course there are files that I created from scratch as well - so whatever name I use should include both (and exclude being made with a camera or scanner). Another way of looking at it is that no "external device" was used in their creation except the computer I'm working on.Because they were "born" on the computer, perhaps CyBorn? :)

[iamback]

Because they were "born" on the computer, perhaps CyBorn? :)I love it! That describes exactly what I'm trying to get at - so I'm going to use that. Thanks!

(Just-upgraded IMatch is now upgrading its database, but once that's done I'll rename the "Computer" directory to "CyBorn". :D)

[terrie]

bo: Remember reading about that man who could tell you what was recorded on an LP by just looking at the grooves, but that wouldn't work with CDs either...LOL!!!

Terrie

[terrie]

cristen: But no matter how logical I attempt to be with folders, those folders alone won't help a search in the future if my need has changedYeah...I'm not sure what the answer is to this unless it's either adding new keywords (seems like the most reasonable approach) which would be "added" to the image when you re-view it/them or starting from scratch which seems to defeat the purpose of DAM entirely...


>>There are simply too many different needs to know exactly what advice to give that would have any meaning for another person. As you said, as many ways to get there as people.

Exactly...I think that any approach that works for the individual is useful (for at least that person) and that there can't be hard and fast rules that say "you must do it this way or you're stupid"...'-}}

Terrie

[ElyseC]

I love it! That describes exactly what I'm trying to get at - so I'm going to use that. Thanks!

(Just-upgraded IMatch is now upgrading its database, but once that's done I'll rename the "Computer" directory to "CyBorn". :D)Aw, shucks, just happy to help! :)

[Cristen Gillespie]

I had a quick look at Portfolio (and a bunch of other programs and reviews) when selecting an application but at least for the Windows platform could not find anything that even matches IMatch. It's very, very powerful, and so far I'm using only a fraction of its capabilities - but you can "grow into" it and not lose any of the meta data you've already stored.

I took a look at iMatch's site and I agree that for the price, it certainly appears to be a good asset manager. I don't think it existed all those years ago when I bought Portfolio (I recall a choice between Cumulus Canto, Thumbs Plus and Portfolio--and Portfolio was the one to offer a student discount), but I'm still happy with the way Portfolio has kept pace with my needs. I'm particularly addicted to the flexibility in their galleries, which keeps being improved upon.

I don't need to return to the original, physical slide, negative or print, so I have no need to name my files according to source. Although a small portion of the originals will be kept for posterity, the vast majority will become digital only. I've named finished files so my family can look at a list and have some idea what to expect, and begun organizing the material on DVDs so they won't have a muddle of people and places. Sometimes thinking of those who may be wanting to find material, not myself, has meant relaxing conventions that would work perfectly well in a library with a librarian in charge, but be meaningless to a relative.

It's not a perfect system yet. I'm still working on it, but I'm spending more of my spare time working on the images than the system. The way I burn the DVDs is the only reason I can get away with being as lax as I've been. Asset management is very time-consuming, unfortunately, and I've little choice but to allot the time to restoration or management, so management gets the shorter shrift.

[Cristen Gillespie]

Terrie: it's either adding new keywords (seems like the most reasonable approach) which would be "added" to the image when you re-view it/them or starting from scratch which seems to defeat the purpose of DAM entirely...

As much as asset management has exploded, most of us who were using a manager umpteen years ago are in the position of having to go back, put finished files back in a folder on the hard drive, add the metadata, then re-burn, or rely on offline "views" of CDs and DVDs. That's enough work to face by itself. I will be swimming in management and adding nothing to the actual body of work if I'm not careful -- which means for me, as I mentioned to Marjolein, being a little more relaxed about being so efficient<G> My family will understand and can further organize the material to suit them when that time comes.

In the meantime, I attempt more efficient ways to use Bridge for adding some metadata quickly (even if I won't have every category covered), using Portfolio to help me edit so I don't waste my time restoring and retouching everything, and using the physical size of a CD or DVD to help act as a "folder" or shoebox of images.

Thank heavens I'm not Getty or Corbis, trying to run a huge stock photo agency<ggg>

[iamback]

As much as asset management has exploded, most of us who were using a manager umpteen years ago are in the position of having to go back, put finished files back in a folder on the hard drive, add the metadata, then re-burn, or rely on offline "views" of CDs and DVDs. That's enough work to face by itself. I will be swimming in management and adding nothing to the actual body of work if I'm not careful -- which means for me, as I mentioned to Marjolein, being a little more relaxed about being so efficient<G>That's a very good point, actually.

I'm in a very different situation, having effectively no digital assets management at all (but by now a good system for archiving my physical - analog - assets). Instead, I'm faced with a fast-increasing set of digital images (mostly used / to be used for web sites), and a need to bring my analog images into the mix.

The latter is, of course, an ugly amount of work, but can be done gradually. And meanwhile I've started to use camphones and now a real (compact) digital camera, increasing the inward flow of digital files. This is definitely a different situation than for someone who has many years worth of digital assets, handled by some sort of assets management software already.

My situation is both an advantage and a disadvantage: the advantage is that I can design a management system from scratch (though for my digital files I've always been reasonably ordered and tidy as such is my nature); the disadvantage is the huge amount of existing analog material I want to have accessible and "publishable" that needs to be - not just managed, but scanned and catalogued in the first place.

One very important feature of IMatch won me over almost immediately: its capability of working with hierarchical "trees" of categories. You don't need to do this - there are many ways of indexing your material in IMatch - but it happens to match my way of thinking perfectly: something I call "brain fit". If you're more happy with keywords (or "tags") or any combination of keywords and categories, you can use that, too, and you can even export IPTC keywords to categories or properties, or vice versa.

I've seen several programs that have some sort of concept of "galleries" (static or dynamic or both); IMatch doesn't have this as such, but has - in addition to the category system - "selections" which can be built on any number of criteria using categories, folders, or variables (including EXIF, IPTC and XMP data). I think that covers all that "galleries" can do and a lot more besides.

It was those capabilities, and their perfect "brain fit" for me, that quickly won me over to IMatch - in addition to very good support from both the developer and peer-to-peer on the IMatch forum; that, combined with a little push from a major goof in the latest Pro version of ThumbsPlus that I'd upgraded to which completely clobbered a set of TIFF files. (Not the most important files - but the experience made TP totally unreliable for me, and I never heard back from the developers when I reported it and offered to send them my files.)

Starting more or less from "scratch" as I am doing is to an extent luxurious (in making choices and setting up a management system) - but the amount of work I'm facing is not! (Major job: travel photography going back to 1987...) Which actually means I have to be efficient or I'll never get anywhere before I drop dead on the stairs of yet another airplane. ;)

But I have no family of my own (except my parents), no descendants - and I feel too "young" still to worry about who will ever "inherit" my material. In a sense that's a luxury, too.

[iamback]

I took a look at iMatch's site and I agree that for the price, it certainly appears to be a good asset manager. I don't think it existed all those years ago when I bought Portfolio (I recall a choice between Cumulus Canto, Thumbs Plus and Portfolio--and Portfolio was the one to offer a student discount), but I'm still happy with the way Portfolio has kept pace with my needs. I'm particularly addicted to the flexibility in their galleries, which keeps being improved upon.I'm pretty sure it didn't exist when I first started looking at graphics software and registered my first version of ThumbsPlus (no student discount). I've never even heard of "Cumulus"... But sometimes you're lucky and the "right" software appears to be there just when you need it. That's what happened for me years ago with HomeSite; finding IMatch (http://www.photools.com/) felt the same: a single, dedicated and very talented developer, who listens to a large group of dedicated users and produces outstanding software. These programs (and their developers) are rare, but happily they still exist and continue to exist in spite of competition from big business and large companies. (I've commented on the concept of "galleries" in my reply to your other message.)

I don't need to return to the original, physical slide, negative or print, so I have no need to name my files according to source. Although a small portion of the originals will be kept for posterity, the vast majority will become digital only.That's something I don't feel comfortable with - not yet anyway. Even though film scanners currently are extremely capable (and I've got a good one), analog material contains data they can't catch and not only am I not giving up on my analog camera(s) yet, I'm not prepared to give up on my analog material either, knowing that prints from those can still surpass anything done with a digital scan from the same material. So for me, the connection needs to be there - which means a careful numbering system, maintained across both analog and digital resources. It's not hard, really - though it took several tries to get right. :)

It's not a perfect system yet. I'm still working on it, but I'm spending more of my spare time working on the images than the system.I think that as long as your material grows, and you continue to work with it, no system is ever perfect.

You can take a picture with a set of specific concepts in your mind, and those will go into categories or keywords when you catalog the result. But you can look at something a week, or many years, later, and see something that you had not considered relevant before - and you need to extend your catalog. If you don't, you'll lose it, because the concepts in your mind keep evolving when your database does not.

[Cristen Gillespie]

Marjolein: I'm faced with a fast-increasing set of digital images (mostly used / to be used for web sites), and a need to bring my analog images into the mix.

This is why every system has to be tailored to the individual/business. If I were trying to keep track of my asssets as a graphic designer, I'd set up my system differently from a wedding & portrait photographer, different from a landscape photographer, and any system for a job would be separate from and different to a personal system. The same software can manage different needs, fortunately.

Marjolein: It was those capabilities, and their perfect "brain fit" for me, that quickly won me over to IMatch

That's certainly what we hope for when we choose software<G>

Marjolein: Starting more or less from "scratch" as I am doing is to an extent luxurious (in making choices and setting up a management system) - but the amount of work I'm facing is not! (Major job: travel photography going back to 1987...) Which actually means I have to be efficient or I'll never get anywhere before I drop dead on the stairs of yet another airplane.

If I were starting out, I'd certainly have paid more attention to IPTC metadata and what categories I was going to need right from the beginning. I wouldn't have to consider going back and updating CDs/DVDs, as I now have to.

Fortunately, asset software can be used by those who are fanatically organized and those who throw everything in a drawer and hope for the best, and help them all<G>

[Cristen Gillespie]

Marjolein: That's something I don't feel comfortable with - not yet anyway. Even though film scanners currently are extremely capable (and I've got a good one), analog material contains data they can't catch and not only am I not giving up on my analog camera(s) yet,

I understand not being comfortable with throwing away lots of data. I wasn't either, at first. I have been through several generations of scanners and know that more data can be extracted as dmax increases.

However, unless I can say that each picture is worth someone taking their time to look at it, it hardly is valuable enough to preserve. I may be the one who wants to look at it again, in which case it lives. Since I'm in the driver's seat here, I can decide.

If I preserve too many pictures, however, I'm guilty of endangering those that are exceptional. They won't be seen. It's up to any editor worth his or her salt to isolate and promote what is worthy and within human endurance to notice.

I have well over a century of family and travel photos, and that doesn't take into account using the camera purely for art, as every member for 4 generations has. There is simply too great an accumulation of physical material for anyone in the next generation to deal with.

The ever cheaper cost of taking photos has created a monster that actually threatens to destroy the very history it promised to preserve--by burying us in too much data.

On the positive side, we at least have choice and instead of preserving forever the one or two pictures of great, great grandmother, we can select among many.

Marjolein: But you can look at something a week, or many years, later, and see something that you had not considered relevant before - and you need to extend your catalog. If you don't, you'll lose it, because the concepts in your mind keep evolving when your database does not.

Very, very true. A catalog is a living, growing thing. Thank heavens for batch actions and templates that let us apply new metadata long after the fact. ;-)

[iamback]

The ever cheaper cost of taking photos has created a monster that actually threatens to destroy the very history it promised to preserve--by burying us in too much data.

On the positive side, we at least have choice and instead of preserving forever the one or two pictures of great, great grandmother, we can select among many.But at the same time historians are lamenting the fact that with the age of digital photography it has not only become easier to take photos - it has also become easier to throw them away, forever. That picture of your great great grandmother that is no longer interesting to you, may in fact be interesting to historians (because of the clothes she was wearing, her hairdo, or other objects in the same photograph).

(OK, so I'm a packrat - but the argument of historians is valid.)

[Cristen Gillespie]

Marjolein: That picture of your great great grandmother that is no longer interesting to you, may in fact be interesting to historians (because of the clothes she was wearing, her hairdo, or other objects in the same photograph).

What they had on and around them isn't lost unless you destroy every picture of great grandma. There is still likely to be more for the historian than ever before (except for those who plant their sole copy on a hard drive -- we can say they fill the role of all those who couldn't afford photography 90 years ago), and more of it relevant since it will be from snapshots of everyday life, not just the photos of studio props and Sunday best -- not always even one's own Sunday best.

However, any historian who truly thinks they want the source material from all the cameras of the past 50 years are kidding themselves. No one in their right mind would want to go through all that carefully enough to preserve what's important and let go of what is redundant, assuming they even know what is important.

For styles, products, and environmental photos, there is more than enough for any historian. The past 70 years have been the most documented in all human history. We're swimming in the stuff.

I understand the packrat impulse. I really do. But it's too non-discriminating to be of value to anyone beyond oneself. If you don't know what you might do with something and it appears useful, of course you keep it. But if you have plenty to represent a time or place, and all this becomes is more, well then, our Governor Brown had a point when he said less is more. <G>

As for my own photos, it's incredibly difficult to keep saying "failure, no there there" and toss, but I would like to think that the occasional photo I make that I deem a success, will be seen by my family, not suffocated under a blanket of the banal. I can't convince myself that every shot I take is of any value whatsoever. All the more reason to manage my assets carefully<G>

[terrie]

cristen: All the more reason to manage my assets carefullyIndeed...I think that one has to be realistic about what constitues an image worth keeping.

Terrie