Reformatting to non-standard NTFS cluster sizes?

Has anyone experimented with non-standard cluster sizes on NTFS
partitions? Typically, the default behaviour is that when creating the
file system, the format utility chooses the ideal cluster size based on
the size of the partition. But typically the highest it ever chooses is
4KB.

http://technet.microsoft.com/en-us/l.../cc767961.aspx

But there are options for choosing upto 64KB, which never get chosen by
default. I have a file system that I calculated has a simple average
file size of 115MB! I was thinking for files that large, I should choose
the highest possible cluster size available.

Yousuf Khan

On Sun, 15 Apr 2012 10:47:48 -0400 "Yousuf Khan" <bbbl67
@spammenot.yahoo.com> wrote in article <4f8adf97$>
>
> Has anyone experimented with non-standard cluster sizes on NTFS
> partitions? Typically, the default behaviour is that when creating the
> file system, the format utility chooses the ideal cluster size based on
> the size of the partition. But typically the highest it ever chooses is
> 4KB.
>
> http://technet.microsoft.com/en-us/l.../cc767961.aspx
>
> But there are options for choosing upto 64KB, which never get chosen by
> default. I have a file system that I calculated has a simple average
> file size of 115MB! I was thinking for files that large, I should choose
> the highest possible cluster size available.
>
> Yousuf Khan

I have partitions with large numbers of enormous .wav files I've
recorded. I have experimented with cluster sizes all the way to 64k but
see very little, if any, change in performance. That surprises me. I also
have several external HDD's (USB- and eSATA-attached) for backup and have
done the same tests with the same result.

On 15/04/2012 12:13 PM, Jason wrote:
> On Sun, 15 Apr 2012 10:47:48 -0400 "Yousuf Khan"<bbbl67
> @spammenot.yahoo.com> wrote in article<4f8adf97$>
>>
>> Has anyone experimented with non-standard cluster sizes on NTFS
>> partitions? Typically, the default behaviour is that when creating the
>> file system, the format utility chooses the ideal cluster size based on
>> the size of the partition. But typically the highest it ever chooses is
>> 4KB.
>>
>> http://technet.microsoft.com/en-us/l.../cc767961.aspx
>>
>> But there are options for choosing upto 64KB, which never get chosen by
>> default. I have a file system that I calculated has a simple average
>> file size of 115MB! I was thinking for files that large, I should choose
>> the highest possible cluster size available.
>>
>> Yousuf Khan
>
> I have partitions with large numbers of enormous .wav files I've
> recorded. I have experimented with cluster sizes all the way to 64k but
> see very little, if any, change in performance. That surprises me. I also
> have several external HDD's (USB- and eSATA-attached) for backup and have
> done the same tests with the same result.

NTFS was originally called HPFS, for High Performance File System. It
was designed to be fast and reliable with low cluster sizes, so as to
reduce blank space. NTFS inherited that ability, so you will see small
differences in performance when you sue larger clusters. Whether those
small diffs are worth it is your call. IMO, the advantage of larger
cluster sizes has been overtaken by improved hardware performance.

HPFS was devised was back the days when MS was creating OS/2 for IBM
(and itself). After the split, MS renamed its version NT. Up to NT 3.5,
both OS/2 and NT could read and write each other's file systems. Up to
Win 2000 there was still a directory labelled OS/2 buried in the
Windows/System tree.

HTH,
Wolf K.

On 4/15/12 1:20 PM, Wolf K wrote:
> On 15/04/2012 12:13 PM, Jason wrote:
>> On Sun, 15 Apr 2012 10:47:48 -0400 "Yousuf Khan"<bbbl67
>> @spammenot.yahoo.com> wrote in article<4f8adf97$>
>>>
>>> Has anyone experimented with non-standard cluster sizes on NTFS
>>> partitions? Typically, the default behaviour is that when creating the
>>> file system, the format utility chooses the ideal cluster size based on
>>> the size of the partition. But typically the highest it ever chooses is
>>> 4KB.
>>>
>>> http://technet.microsoft.com/en-us/l.../cc767961.aspx
>>>
>>> But there are options for choosing upto 64KB, which never get chosen by
>>> default. I have a file system that I calculated has a simple average
>>> file size of 115MB! I was thinking for files that large, I should choose
>>> the highest possible cluster size available.
>>>
>>> Yousuf Khan
>>
>> I have partitions with large numbers of enormous .wav files I've
>> recorded. I have experimented with cluster sizes all the way to 64k but
>> see very little, if any, change in performance. That surprises me. I also
>> have several external HDD's (USB- and eSATA-attached) for backup and have
>> done the same tests with the same result.
>
> NTFS was originally called HPFS, for High Performance File System. It
> was designed to be fast and reliable with low cluster sizes, so as to
> reduce blank space. NTFS inherited that ability, so you will see small
> differences in performance when you sue larger clusters. Whether those
> small diffs are worth it is your call. IMO, the advantage of larger
> cluster sizes has been overtaken by improved hardware performance.
>
> HPFS was devised was back the days when MS was creating OS/2 for IBM
> (and itself). After the split, MS renamed its version NT. Up to NT 3.5,
> both OS/2 and NT could read and write each other's file systems. Up to
> Win 2000 there was still a directory labelled OS/2 buried in the
> Windows/System tree.

My question would be, does it affect the amount of data that can be stored?

Logically, the smaller the cluster size, it would require more entries
on the disk to keep track of which cluster belonged to which file, and
the order of the clusters. Larger sized clusters would require fewer
entries.

If your files are relatively small, I wouldn't think there'd be much
difference. But, what if you have huge amounts of large and larger
video, graphic, and audio files?


--
Ken

Mac OS X 10.6.8
Firefox 11.0
Thunderbird 11.0.1
LibreOffice 3.5.1.2

In article <>, Jason <> wrote:
>On Sun, 15 Apr 2012 10:47:48 -0400 "Yousuf Khan" <bbbl67
>@spammenot.yahoo.com> wrote in article <4f8adf97$>
>>
>> Has anyone experimented with non-standard cluster sizes on NTFS
>> partitions? Typically, the default behaviour is that when creating the
>> file system, the format utility chooses the ideal cluster size based on
>> the size of the partition. But typically the highest it ever chooses is
>> 4KB.
>>
>> http://technet.microsoft.com/en-us/l.../cc767961.aspx
>>
>> But there are options for choosing upto 64KB, which never get chosen by
>> default. I have a file system that I calculated has a simple average
>> file size of 115MB! I was thinking for files that large, I should choose
>> the highest possible cluster size available.
>>
>> Yousuf Khan
>
>I have partitions with large numbers of enormous .wav files I've
>recorded. I have experimented with cluster sizes all the way to 64k but
>see very little, if any, change in performance. That surprises me. I also
>have several external HDD's (USB- and eSATA-attached) for backup and have
>done the same tests with the same result.
>
>
It really has nothing to do with differences in performance, and more to do
with if you choose the wrong cluster size for the files you intend to store,
but the more wasted space per file.
Youd want a smaller clsuetr size if you were storing thousands of small MIDI
files, and larger sized clusters for large backup files or DV video captures.

"Ken Springer" <> wrote in message
news:jmf9qj$629$...
> On 4/15/12 1:20 PM, Wolf K wrote:
>> On 15/04/2012 12:13 PM, Jason wrote:
>>> On Sun, 15 Apr 2012 10:47:48 -0400 "Yousuf Khan"<bbbl67
>>> @spammenot.yahoo.com> wrote in article<4f8adf97$>
>>>>
>>>> Has anyone experimented with non-standard cluster sizes on NTFS
>>>> partitions? Typically, the default behaviour is that when creating the
>>>> file system, the format utility chooses the ideal cluster size based on
>>>> the size of the partition. But typically the highest it ever chooses is
>>>> 4KB.
>>>>
>>>> http://technet.microsoft.com/en-us/l.../cc767961.aspx
>>>>
>>>> But there are options for choosing upto 64KB, which never get chosen by
>>>> default. I have a file system that I calculated has a simple average
>>>> file size of 115MB! I was thinking for files that large, I should
>>>> choose
>>>> the highest possible cluster size available.
>>>>
>>>> Yousuf Khan
>>>
>>> I have partitions with large numbers of enormous .wav files I've
>>> recorded. I have experimented with cluster sizes all the way to 64k but
>>> see very little, if any, change in performance. That surprises me. I
>>> also
>>> have several external HDD's (USB- and eSATA-attached) for backup and
>>> have
>>> done the same tests with the same result.
>>
>> NTFS was originally called HPFS, for High Performance File System. It
>> was designed to be fast and reliable with low cluster sizes, so as to
>> reduce blank space. NTFS inherited that ability, so you will see small
>> differences in performance when you sue larger clusters. Whether those
>> small diffs are worth it is your call. IMO, the advantage of larger
>> cluster sizes has been overtaken by improved hardware performance.
>>
>> HPFS was devised was back the days when MS was creating OS/2 for IBM
>> (and itself). After the split, MS renamed its version NT. Up to NT 3.5,
>> both OS/2 and NT could read and write each other's file systems. Up to
>> Win 2000 there was still a directory labelled OS/2 buried in the
>> Windows/System tree.
>
> My question would be, does it affect the amount of data that can be
> stored?
>
> Logically, the smaller the cluster size, it would require more entries on
> the disk to keep track of which cluster belonged to which file, and the
> order of the clusters. Larger sized clusters would require fewer entries.
>
> If your files are relatively small, I wouldn't think there'd be much
> difference. But, what if you have huge amounts of large and larger video,
> graphic, and audio files?

But those drives by definition arent likely to be where you are obsessed
about space use.

On 15/04/2012 12:13 PM, Jason wrote:
> I have partitions with large numbers of enormous .wav files I've
> recorded. I have experimented with cluster sizes all the way to 64k but
> see very little, if any, change in performance. That surprises me. I also
> have several external HDD's (USB- and eSATA-attached) for backup and have
> done the same tests with the same result.

Well, I have gone ahead done the reformat to one my partitions, I'm now
using 64K clusters on that. I'm in the middle of some rearrangement of
data on my hard drives. So what I've been going around doing is copying
and moving entire file systems worth of data from one drive to another.
When I did the transfer between a default 4K cluster drive and a 64K
cluster drive, the speed of transfer is not hugely different. However,
there was a huge difference in responsiveness of the drives. I was
watching the transfers (which lasted a couple of hours each) on the
Windows 7 Resource Monitor, and you could see that the 4K drives were
absolutely pegged at 100% disk activity, with disk queues that were
hovering >=1.0. But the drive that was using 64K clusters had disk
activity of 40-50%, and disk queues well <1.0. This worked in both
directions, whether I was copying off of the 64K drive, or copying to
the 64K drive, it was the same way.

Yousuf Khan

"GMAN" <> wrote in message
news:EiGir.179666$...
> In article <>, Jason
> <> wrote:
>>On Sun, 15 Apr 2012 10:47:48 -0400 "Yousuf Khan" <bbbl67
>>@spammenot.yahoo.com> wrote in article <4f8adf97$>
>>>
>>> Has anyone experimented with non-standard cluster sizes on NTFS
>>> partitions? Typically, the default behaviour is that when creating the
>>> file system, the format utility chooses the ideal cluster size based on
>>> the size of the partition. But typically the highest it ever chooses is
>>> 4KB.
>>>
>>> http://technet.microsoft.com/en-us/l.../cc767961.aspx
>>>
>>> But there are options for choosing upto 64KB, which never get chosen by
>>> default. I have a file system that I calculated has a simple average
>>> file size of 115MB! I was thinking for files that large, I should choose
>>> the highest possible cluster size available.
>>>
>>> Yousuf Khan
>>
>>I have partitions with large numbers of enormous .wav files I've
>>recorded. I have experimented with cluster sizes all the way to 64k but
>>see very little, if any, change in performance. That surprises me. I also
>>have several external HDD's (USB- and eSATA-attached) for backup and have
>>done the same tests with the same result.
>>
>>
> It really has nothing to do with differences in performance, and more to
> do
> with if you choose the wrong cluster size for the files you intend to
> store,
> but the more wasted space per file.
> Youd want a smaller clsuetr size if you were storing thousands of small
> MIDI
> files, and larger sized clusters for large backup files or DV video
> captures.

But again, with that last situation, you are likely to be choosing between
2TB and
3TB drives base on $/GB and the small saving in free space isnt likely to
matter.
All that really does is see you be able to squeeze one more file on the
drive at most.

On 15/04/2012 4:08 PM, Ken Springer wrote:
>
> My question would be, does [cluster size] affect the amount of data that can be stored?

Yes. The smaller the cluster size, the less wasted space.

Suppose a file is 87 K. With 4K clusters, that will take 22 clusters ==>
88K space for 87 K data. With 64K clusters, you'll need 2 ==> 128K
space for 87K data.

Now suppose your documents average about 87K in size....

HTH,
Wolf K.

On 15/04/2012 4:08 PM, Ken Springer wrote:
> My question would be, does it affect the amount of data that can be stored?
>
> Logically, the smaller the cluster size, it would require more entries
> on the disk to keep track of which cluster belonged to which file, and
> the order of the clusters. Larger sized clusters would require fewer
> entries.

You're thinking of the FAT system which has a global bitmap of clusters
assigned to files and folders. The bigger the drive and the smaller the
cluster size, the bigger the bitmap gets. In NTFS, it's more of an
extent-based system, which basically contains a starting and ending
cluster address of all extents belonging to a file. The metadata on the
disk only grows with the number of files, but not necessarily with the
size of the file system. So the size of the clusters won't make a
difference to the size of the metadata, just the number of files you put
into it.

> If your files are relatively small, I wouldn't think there'd be much
> difference. But, what if you have huge amounts of large and larger
> video, graphic, and audio files?

However, you will get more slack with small files in a large-cluster
filesystem. Even the smallest files will take up a minimum of 64K with a
filesystem with 64K clusters.

However, I've found that the responsiveness of a large-cluster-size
filesystem, is much greater when you also have large files.

Yousuf Khan

Yousuf Khan