Advice on RAW capture size

Printable View

11-01-2012, 10:11 PM
EricPvpi

Advice on RAW capture size

So...I made the leap into DSLR this year and capturing lots of pictures. I have read plenty of JPG verse RAW threads and moved to just shooting RAW.

Now, my library is growing quite large in size, probably sitting at 50GB for what I shot on my 60D this year, which is what I have for a 10 years of JPG's.

Question - what are tradeoffs, recommendations for Large verse Medium Raw? I am shooting with 60D purely for personal use. Pictures would be mainly for online or 8x10 at the largest. When is one the better choice over the other.

I am not really concerned about local storage. I use Lightroom for management and store locally on a network storage device which has plenty of room and easy to expand. I really start thinking about the implications for backup, especially if I want to use an online backup service. The costs could really shot up for ones based on total storage.

So, is large RAW what I should be using and I should figure out storage or is it overkill for my use? Certainly, I don't want to drop lower than I need since I can't get it back.

Or is medium RAW the right one for my use? This save a little space and a bit easier on the computer to process.

Or is it a mix? Use medium most of the time, large when I know I will want to crop. I want to keep it somewhat simple, otherwise I will forget when I need to switch.

Thanks for letting me ask such a simple question and would appreciate anyone's experience.

Thanks,
Eric

Large RAW - 17.9 megapixal, 24.5 MB filesize, (5184 x 3456)
Medium RAW - 10.1 megapixal, 16.7 MB filesize, (3888 x 2592)
11-01-2012, 10:50 PM
Daniel Browning

Quote:

Originally Posted by EricPvpi

Question - what are tradeoffs, recommendations for Large verse Medium Raw?

Well, first of all, they're not raw. Canon just calls it that for marketing purposes. Technically, they are in JPEG file format, but they are more similar to a TIFF. They have some JPEG compression -- that is, half the color information is thrown away (this is *after* throwing away a lot of the resolution by the in-camera demosaic and downsample). But it's pretty light compression and most of the time will go completely unnoticed.

The pros are obvious: smaller files, more shots per card, less backup space required, faster processing, etc. Some pros that you might expect, but do not always get, are things like longer high-FPS shooting streaks before hitting the buffer limit. (It doesn't always work out, because the camera is still doing all the normal processing.)

The cons are more nuanced. Many photographers have what I consider to be a poor perception of the value of resolution. I wrote more about it here a long time ago, but I can't find the thread. Here is a thread from a different forum:

http://photography-on-the.net/forum/...d.php?t=747749

In addition to the loss of resolution, there is the loss of the ability to demosaic the file. Raw converters have untold manhours poured into their demosaic algorithms for optimizing resolution, color, noise, etc. There is a wide variety of them and they get better every year. You can go back to raw files you took 5 years ago and extract more detail, better color, etc. But not if you used sRAW or mRAW. The demosaic and downsample has already been applied, so the only thing the raw converter can do is the same things you can do to a TIFF file: play with the colors, etc. Maybe you're happy with the in-camera demosaic now, but what if your taste has changed in a few years?

Quote:

Originally Posted by EricPvpi

So, is large RAW what I should be using

Yes.

Quote:

Originally Posted by EricPvpi

or is it overkill for my use?

Yes and no. Only you can decide if the hassle of increased file sizes is worth it, but I would argue that in your case it is. But what's *really* frustrating about all this is that you shouldn't *have* to decide. Canon certainly could use much more intelligent and less harmful compression that truly is raw, but they don't. (Partly it's our fault, since most of us photographers think all compression is evil, even though we readily use things that are much more harmful than compression.)

I discussed some of the superior alternatives to sRAW/mRAW here:

http://photography-on-the.net/forum/...d.php?t=730030

Quote:

Originally Posted by EricPvpi

Large RAW - 17.9 megapixal, 24.5 MB filesize, (5184 x 3456)

Consider yourself lucky. My D800 raws average ~45MP.
11-01-2012, 11:00 PM
EricPvpi

Thanks, I will read through the other postings. One reason I started shooting large raw was I wanted to shoot for my future ability of post processing rather than what I know now. I want to capture enough that I can go back and tweak the pictures I really like as I learn more.
11-01-2012, 11:00 PM
andnowimbroke

What if you converted to dng for archive? Does stuff get lost if you want to go back and pull more info out of it. I know if you use dng, the file size will shrink, but if you use dng on an sRAW, it will increase in size because of how the info was being stored.
11-01-2012, 11:21 PM
Daniel Browning
Quote:

Originally Posted by andnowimbroke

What if you converted to dng for archive?

There are several possible issues here:
- All software has bugs. If you lose your original, the DNG may not have all the same data due to a bug in the converter.
- Some raw converters treat originals and DNG files differently. So even if there is no bug, the results may come out different even if they have the exact same data and metadata.
Quote:

Originally Posted by andnowimbroke

Does stuff get lost if you want to go back and pull more info out of it.

It your DNG converter doesn't have any bugs, and your raw converter treats both files the exact same, then you will get the same results from the DNG as from the original.
11-01-2012, 11:21 PM
EricPvpi

For me, I don't think I would go through the process of converting for archive. I would rather store to the NAS drive as raw and set my backup to backup from that location. If I have a step in the middle, I will get lazy and not do it :)
11-01-2012, 11:59 PM
Busted Knuckles

I quick trip to best buy for USB drives are $100 per terabyte - storage is cheap a 1 tb drive would take 20 years to fill up at your rate of usage.

Quote:

Originally Posted by EricPvpi

For me, I don't think I would go through the process of converting for archive. I would rather store to the NAS drive as raw and set my backup to backup from that location. If I have a step in the middle, I will get lazy and not do it :)
11-02-2012, 12:30 AM
HDNitehawk

Go with full Raw.

In a year or so online backup will be cost and time prohibitive.
I have four years worth of RAW files saved now and have 1.7 TB in files.
External hard drives are cheap and you are a long way from having storage issues.
11-02-2012, 01:36 AM
EricPvpi

Quote:

Originally Posted by Busted Knuckles

I quick trip to best buy for USB drives are $100 per terabyte - storage is cheap a 1 tb drive would take 20 years to fill up at your rate of usage.

Quote:

Originally Posted by HDNitehawk

Go with full Raw.

In a year or so online backup will be cost and time prohibitive.
I have four years worth of RAW files saved now and have 1.7 TB in files.
External hard drives are cheap and you are a long way from having storage issues.

Thanks. I was trying to figure out my off-premise backup and realized online might be costly based on what I have now. With full raw it will add up. I have a mirrored 1 TB NAS drive as my main storage and as that fills, I can easily swap those out to 2 TB, so main storage is good and handles hardware failures.

Online backup would have been easier since once it is setup it is automated. I have some external drives, that is just more work to manage doing the backups and getting them offsite.

Carbonite says it has unlimited backup. You have to get the pricier plan for external drives, I just need to research if they exclude raw or start having limitations in the fine print. I am willing to pay a little more if it is automated and hands off. My worry with external drives is the discipline to keep up with it and not have them sitting in the house in the event of fire, flood, etc.

Eric
11-02-2012, 03:08 AM
HDNitehawk

There was a thread a while back about this. One of the things that was discovered is that the online backup was way to slow.
Second is that if you are like me you do not have enough room on your hard drive for all your files, so you need a service that will do your other drives.
I thought it would be convenient to have an online service, all I checked in to that were unlimited were reported to be to slow or had other issues. Those that would be fast enough were cost prohibitive. If you find a good option let us know on the forum.
11-02-2012, 12:11 PM
EricPvpi

Quote:

Originally Posted by HDNitehawk

There was a thread a while back about this. One of the things that was discovered is that the online backup was way to slow.
Second is that if you are like me you do not have enough room on your hard drive for all your files, so you need a service that will do your other drives.
I thought it would be convenient to have an online service, all I checked in to that were unlimited were reported to be to slow or had other issues. Those that would be fast enough were cost prohibitive. If you find a good option let us know on the forum.

I think I read that one and made a comment about the online service I was going to use. It was rated well, but not unlimited. Once I started adding up what I had so I could sign up, I realized I will quickly hit the limits, so I started this one to make sure I am capturing at the right level for my needs.

I am an IT guy, so I am supposed to be better at backing up. It is embarrassing when we lose data :)
11-02-2012, 03:31 PM
Daniel Browning

Quote:

Originally Posted by HDNitehawk

Those that would be fast enough were cost prohibitive. If you find a good option let us know on the forum.

I use fdcservers.net. $45/month ($200 setup) for a dedicated Atom server, 2TB hdd, and unlimited 10 Mbps connection. It takes me about 3 weeks to transfer 2TB, but after it transferred the first time, all I have to do is send changes. I use Duplicity to store the backups encrypted. But this is a highly technical solution that would be very difficult for normal computer users to setup.
11-04-2012, 04:24 AM
Sinh Nhut Nguyen

I separate my shootings into two categories, Art and family snapshot. For art I shoot RAW and for family snapshot I shoot Medium Raw. I like to have the flexible of RAW files and also save space, download and upload times therefore I choose Medium Raw. Heck, I should shoot Large JPEG for family snapshots because I rarely edit anything. 95% of the family snapshot will go straight to Facebook and the rest will be printed in 4x6 so a 10 megapixel file is more than what you need.
11-04-2012, 12:00 PM
EricPvpi

Quote:

Originally Posted by HDNitehawk

There was a thread a while back about this. One of the things that was discovered is that the online backup was way to slow.
Second is that if you are like me you do not have enough room on your hard drive for all your files, so you need a service that will do your other drives.
I thought it would be convenient to have an online service, all I checked in to that were unlimited were reported to be to slow or had other issues. Those that would be fast enough were cost prohibitive. If you find a good option let us know on the forum.

Quote:

Originally Posted by Daniel Browning

I use fdcservers.net. $45/month ($200 setup) for a dedicated Atom server, 2TB hdd, and unlimited 10 Mbps connection. It takes me about 3 weeks to transfer 2TB, but after it transferred the first time, all I have to do is send changes. I use Duplicity to store the backups encrypted. But this is a highly technical solution that would be very difficult for normal computer users to setup.

Looking at Carbonite, which is unlimited, it would take 25+ days for the initial upload. And I have the feeling it won't let me backup from my NAS drive since I see they distinguish between external drives and NAS drives.

Right now I am thinking about upgrading my NAS drive to a newer version. For around $750 I can buy 2 Netgear ReadyNAS duo v2 drives, each with mirrored 2 TB drives. You use 1 as your primary drive at home and then replicate it to the second drive. You can then place the second drive at another location and keep them in sync.

Besides the backup solution, it gives you other features http://www.netgear.com/landing/personalcloud/ such as remote access, dropbox like sync, and photo sharing. I have the previous version of the Netgear which can't do the replication, but I have liked keeping all of my data on the NAS drive since it keeps it independent of each computer as well as being mirrored. It makes it a lot easier when a computer has trouble or gets replaced.

So, $375 gets you a 2 TB home storage solution with another $375 as a one time cost for a backup solution. Depending on how you configure the NAS, you can go up to 6 GB in each, so you can grow it if needed. Best part is once it is setup, it will do the backup to the second unit on its own.

Eric