Performance issue?

Yesterday evening I discovered that I have not getting the backup that
I want out of SyncToy. To facilitate working on the problem I decided
to offload the current backup to a spare 1 TB drive in the same box. I
created a folder called Drive E, did a control A select in Win 7
Explorer, click on Copy, and select folder Drive, click on Paste and
here we are 19 hours later waiting for 250 G to be transfered.

Both drives are Seagate SATA on the same MSI MB. Is this normal?

On 9/19/2011 7:04 PM, W8CCW wrote:
> Yesterday evening I discovered that I have not getting the backup that
> I want out of SyncToy. To facilitate working on the problem I decided
> to offload the current backup to a spare 1 TB drive in the same box. I
> created a folder called Drive E, did a control A select in Win 7
> Explorer, click on Copy, and select folder Drive, click on Paste and
> here we are 19 hours later waiting for 250 G to be transfered.
>
> Both drives are Seagate SATA on the same MSI MB. Is this normal?
>

"Is this normal?"
Not really.
I'd try a different method to copy.

Part of the issue may have to do with what SATA controller is used.
If both drives use the same controller, that can slow things down.
Another possibility has to do with how much space is available on the HD
drive used to "buffer" the copy, and the amount of installed RAM
available for the copy process.
Finally, all Windows copy methods are not equal in speed. (Why, I never
bothered to find out.)

W8CCW wrote:
> Yesterday evening I discovered that I have not getting the backup that
> I want out of SyncToy. To facilitate working on the problem I decided
> to offload the current backup to a spare 1 TB drive in the same box. I
> created a folder called Drive E, did a control A select in Win 7
> Explorer, click on Copy, and select folder Drive, click on Paste and
> here we are 19 hours later waiting for 250 G to be transfered.
>
> Both drives are Seagate SATA on the same MSI MB. Is this normal?
>

To "bound" or tell you how bad a drive can be, consider the transfer
rate achieved when you defragment a drive.

One of my OSes, does that at around 1MB/sec. My best case is around 3MB/sec
(on another computer, with a different Windows OS). The reason those speeds
happen, is the transfer size is tiny, and there is a lot of head movement
per megabyte of data. During defrag, the disk head flies around like crazy.

If your drive has many small files, transfer rates are going to suck just
like defragmenting does.

If the drive was say, your "movie" drive, and there were a relatively
small number of gigabyte sized files, then something is seriously wrong.
On the one hand, it's possible to have extreme fragmentation. One way
to achieve that, would be to have a quad TV tuner in the PC, and record four
TV programs at the same time. That could cause the four, gigabyte sized
recorded files, to have thousands of fragments. Copying severely fragmented
files (just freshly recorded), would suck in the same way as our small file
example would, because the disk head has to move to find every fragment.
Smooth contiguous files, written with no other interference at the time,
can transfer at 125MB/sec+.

What I use for disk to disk transfer, is Robocopy. I use the command line,
but there is also a GUI. Note that, that tool is extremely dangerous, when
doing things like the "mirror" option. Mirror would erase any destination
folder contents not needed to mirror the source disk. If you accidentally
point the mirror operation at the wrong drive, you lose the contents
of the destination drive. I've had just one accident with Robocopy so
far, and had to modify my procedures a bit so it won't happen again.

http://technet.microsoft.com/en-us/m...spotlight.aspx

Robocopy does non-blocking I/O, and can issue read and write requests at
the same point in time (to two different drives). So it can overlap
commands. I find it generally good, and achieves an average of perhaps 40MB/sec
with my crappy $50 drives here. That's an average - transfer rate can vary
from 3MB/sec (doing the small files again) to 125MB/sec (copying my pagefile
or hiberfile on an OS partition). The long term average while copying
thousands of files, is in the 40MB/sec ballpark. If I copy A-->B then
B-->A, I usually get a *different* average value - don't ask me why.
That happened even when disk A is exactly the same model number as disk B.

If we do the math for you, 250GB is 250000MB in 19*60*60 seconds or
250000/68400 = 3.65MB/sec, which would be fine if you had a million
small files on the drive. You may find Robocopy does a little better -
I think it's a lot better than the file system copy routine, for
comparison.

Just for the record, this is a typical command line I use for moving the
entire contents from one partition to another. I run this in an MSDOS
(command prompt) window. I keep a logfile for each transfer, mainly
to compare the numbers at the end of the log. If copying a partition off
to another partition, then back again, I like to verify I got all the files.

robocopy L:\ E:\ /mir /copy:datso /dcopy:t /r:3 /w:2 /zb /np /tee /v /log:robocopy_ell_to_e.log

Typical log results:

****************************************
-------------------------------------------------------------------------------
ROBOCOPY :: Robust File Copy for Windows :: Version XP026
-------------------------------------------------------------------------------
Started : Tue Sep 06 20:23:59 2011
Source = L:\
Dest = E:\
Files : *.*
Options : *.* /V /TEE /S /E /COPYATSO /DCOPY:T /PURGE /MIR /ZB /NP /R:3 /W:2
------------------------------------------------------------------------------

NOTE : NTFS Security may not be copied - Source may not be NTFS.
NOTE : NTFS Security may not be copied - Destination may not be NTFS.

26 L:\
same 2.9 g hiberfil.sys
same 2.0 g pagefile.sys
...

Total Copied Skipped Mismatch FAILED Extras
Dirs : 14376 14375 1 0 0 0
Files : 178733 178731 2 0 0 0
Bytes : 39.895 g 34.896 g 4.999 g 0 0 0
Times : 0:29:50 0:16:06 0:00:00 0:13:44

Speed : 38778320 Bytes/sec. <--- my long term average copy rate
Speed : 2218.913 MegaBytes/min.

Ended : Tue Sep 06 20:53:50 2011
****************************************

Paul

On 19/09/2011 7:04 PM, W8CCW wrote:
> Yesterday evening I discovered that I have not getting the backup that
> I want out of SyncToy. To facilitate working on the problem I decided
> to offload the current backup to a spare 1 TB drive in the same box. I
> created a folder called Drive E, did a control A select in Win 7
> Explorer, click on Copy, and select folder Drive, click on Paste and
> here we are 19 hours later waiting for 250 G to be transfered.
>
> Both drives are Seagate SATA on the same MSI MB. Is this normal?


Use Resource Monitor and watch the Disk tab. Under the Storage heading
you'll see all of your drive letters listed. Make sure that disk queue
length on any one of them don't go over 1.00 during the transfer.

Yousuf Khan

I started a copy of the "documents" item as displayed in Power Desk 8
from C: to E: . It is a lengthy collection but I am not sure what it
is really copying. Whatever, it needs back up.

I don't think the displayed file system accurately displays the
physical file system. Any comments?


On Mon, 19 Sep 2011 19:04:51 -0400, W8CCW <>
wrote:

>Yesterday evening I discovered that I have not getting the backup that
>I want out of SyncToy. To facilitate working on the problem I decided
>to offload the current backup to a spare 1 TB drive in the same box. I
>created a folder called Drive E, did a control A select in Win 7
>Explorer, click on Copy, and select folder Drive, click on Paste and
>here we are 19 hours later waiting for 250 G to be transfered.
>
>Both drives are Seagate SATA on the same MSI MB. Is this normal?

W8CCW wrote:
> I started a copy of the "documents" item as displayed in Power Desk 8
> from C: to E: . It is a lengthy collection but I am not sure what it
> is really copying. Whatever, it needs back up.
>
> I don't think the displayed file system accurately displays the
> physical file system. Any comments?

I would agree with that sentiment.

After just about bricking my Windows 7 laptop yesterday, I'd recommend
you at least run the "full system backup" option built into Windows 7.
That creates a couple .vhd files, and you can find them by looking for
..vhd on your hard drive. My C: makes a 26GB .vhd file. After I've run
a complete system backup, I copy the two .vhd files to another
drive (for a rainy day). The .vhd files can be randomly accessed,
if you need to get at individual files.

That copy, has as much stuff as it is safe to copy. It would be
more complete than other backup methods.

Since I had that available to me (only two days old), I immediately
used that to clean up the mess I'd made.

It's really like tromping around in a mine field... Beware.

Who knows. Maybe when you create those .vhd files, the
copy operation will go faster ?

http://www.intowindows.com/how-to-cr...-in-windows-7/

http://windows.microsoft.com/en-US/w...m-image-backup

Another piece of trivia I learned in the last couple days. If the
"Set up backup" button doesn't work (which you wouldn't be using
in this case anyway), the command line equivalent of clicking
that button, is

sdclt.exe /configure

A number of my backup buttons don't work right (thanks to the Acer
added software...), but at least I've got a way to access that one.

http://answers.microsoft.com/en-us/w...8-1b92a663f753

HTH,
Paul

> On Mon, 19 Sep 2011 19:04:51 -0400, W8CCW <>
> wrote:
>
>> Yesterday evening I discovered that I have not getting the backup that
>> I want out of SyncToy. To facilitate working on the problem I decided
>> to offload the current backup to a spare 1 TB drive in the same box. I
>> created a folder called Drive E, did a control A select in Win 7
>> Explorer, click on Copy, and select folder Drive, click on Paste and
>> here we are 19 hours later waiting for 250 G to be transfered.
>>
>> Both drives are Seagate SATA on the same MSI MB. Is this normal?

On Tue, 20 Sep 2011 20:11:10 -0400, Paul <> wrote:

>After just about bricking my Windows 7 laptop yesterday, I'd recommend
>you at least run the "full system backup" option built into Windows 7.
>That creates a couple .vhd files, and you can find them by looking for
>.vhd on your hard drive. My C: makes a 26GB .vhd file. After I've run
>a complete system backup, I copy the two .vhd files to another
>drive (for a rainy day). The .vhd files can be randomly accessed,
>if you need to get at individual files.

I'm curious as to why you do a backup and then copy the image files to
another drive. Why not back up directly to that other drive? Do you
keep a .vhd handy as sort of a filestore and put away a second copy of
that same image for safe keeping?

>That copy, has as much stuff as it is safe to copy. It would be
>more complete than other backup methods.

I'd say it's equally complete as other image backups, no? Or what did
you mean?

<snip>

--

Char Jackson

Char Jackson wrote:
> On Tue, 20 Sep 2011 20:11:10 -0400, Paul <> wrote:
>
>> After just about bricking my Windows 7 laptop yesterday, I'd recommend
>> you at least run the "full system backup" option built into Windows 7.
>> That creates a couple .vhd files, and you can find them by looking for
>> .vhd on your hard drive. My C: makes a 26GB .vhd file. After I've run
>> a complete system backup, I copy the two .vhd files to another
>> drive (for a rainy day). The .vhd files can be randomly accessed,
>> if you need to get at individual files.
>
> I'm curious as to why you do a backup and then copy the image files to
> another drive. Why not back up directly to that other drive? Do you
> keep a .vhd handy as sort of a filestore and put away a second copy of
> that same image for safe keeping?
>
>> That copy, has as much stuff as it is safe to copy. It would be
>> more complete than other backup methods.
>
> I'd say it's equally complete as other image backups, no? Or what did
> you mean?
>
> <snip>
>

The .vhd is an exact copy, as exact as they can make it. It's used
for doing P2V conversions, so it has to be able to handle any trick
present in NTFS. It should also use only as many sectors as are needed
to represent the file system.

As to why the backup was on my data partition on the laptop, that
was a "happy accident". When I made that backup, I was actually testing
the various "buttons" in my System Protection window (the window
partially broken by Acer). That one happened to work, so I said,
what the hell, let it run to completion, as I have room on the data partition.
And I also wanted to test, whether the data partition could get damaged or not,
in a later experiment. That file was going to be part of my "test data" set.

Little did I know, that later I would actually *need* that backup :-)

So it wasn't an actual part of a backup strategy, just another
one of my stupid experiments. I'm a "learn by breaking things"
kind of guy.

Paul

On Tue, 20 Sep 2011 21:53:59 -0400, Paul <> wrote:

>Char Jackson wrote:
>> On Tue, 20 Sep 2011 20:11:10 -0400, Paul <> wrote:
>>
>>> After just about bricking my Windows 7 laptop yesterday, I'd recommend
>>> you at least run the "full system backup" option built into Windows 7.
>>> That creates a couple .vhd files, and you can find them by looking for
>>> .vhd on your hard drive. My C: makes a 26GB .vhd file. After I've run
>>> a complete system backup, I copy the two .vhd files to another
>>> drive (for a rainy day). The .vhd files can be randomly accessed,
>>> if you need to get at individual files.
>>
>> I'm curious as to why you do a backup and then copy the image files to
>> another drive. Why not back up directly to that other drive? Do you
>> keep a .vhd handy as sort of a filestore and put away a second copy of
>> that same image for safe keeping?
>>
>>> That copy, has as much stuff as it is safe to copy. It would be
>>> more complete than other backup methods.
>>
>> I'd say it's equally complete as other image backups, no? Or what did
>> you mean?
>>
>> <snip>
>>
>
>The .vhd is an exact copy, as exact as they can make it. It's used
>for doing P2V conversions, so it has to be able to handle any trick
>present in NTFS. It should also use only as many sectors as are needed
>to represent the file system.

Ok, sounds like every other image backup. Good to know.

>As to why the backup was on my data partition on the laptop, that
>was a "happy accident". When I made that backup, I was actually testing
>the various "buttons" in my System Protection window (the window
>partially broken by Acer). That one happened to work, so I said,
>what the hell, let it run to completion, as I have room on the data partition.
>And I also wanted to test, whether the data partition could get damaged or not,
>in a later experiment. That file was going to be part of my "test data" set.
>
>Little did I know, that later I would actually *need* that backup :-)
>
>So it wasn't an actual part of a backup strategy, just another
>one of my stupid experiments. I'm a "learn by breaking things"
>kind of guy.

Got it. Thanks for indulging me.

Any relation to Jeff Liebermann over in alt.internet.wireless? His
motto (and his website) is "Learn By Destroying".

--

Char Jackson

Thank you,
It looks like we are in the same situation.

The xcopy in Dos seemed to whistle right along until I got to a file
that was created by Thunderbird. The current release of Tbird does not
always display attachments as I expect. Those are necessary to
save/view and dispose. I created a directory named "1" in the Pictures
Folder to deal with these files. This particular file name was "Mom, I
gotta pee." The length was 0 bytes. Explorer would not delete it. Dos
saw it with as "Mom,"
Dos did allow me to delete the directory "1"
When I looked at the Pictures folder with Win Explorer the "1" folder
was still there! It did delete OK with Win Explorer.

I have now put back my "1" folder in the "Pictures" folder. I may need
to do this again. I will start a fresh xcopy later today. I am using
the command line of C:\User\John\xcopy /s /e on the E:drive and it
looks do be working.

Although the directory "Users" is accessible from the command line it
does not appear in WinExplorer.

If this Usenet group did not exist I would not have a clue as to where
to find useful advice on these problems.


On Tue, 20 Sep 2011 20:11:10 -0400, Paul <> wrote:

>W8CCW wrote:
>> I started a copy of the "documents" item as displayed in Power Desk 8
>> from C: to E: . It is a lengthy collection but I am not sure what it
>> is really copying. Whatever, it needs back up.
>>
>> I don't think the displayed file system accurately displays the
>> physical file system. Any comments?
>
>I would agree with that sentiment.
>
>After just about bricking my Windows 7 laptop yesterday, I'd recommend
>you at least run the "full system backup" option built into Windows 7.
>That creates a couple .vhd files, and you can find them by looking for
>.vhd on your hard drive. My C: makes a 26GB .vhd file. After I've run
>a complete system backup, I copy the two .vhd files to another
>drive (for a rainy day). The .vhd files can be randomly accessed,
>if you need to get at individual files.
>
>That copy, has as much stuff as it is safe to copy. It would be
>more complete than other backup methods.
>
>Since I had that available to me (only two days old), I immediately
>used that to clean up the mess I'd made.
>
>It's really like tromping around in a mine field... Beware.
>
>Who knows. Maybe when you create those .vhd files, the
>copy operation will go faster ?
>
>http://www.intowindows.com/how-to-cr...-in-windows-7/
>
>http://windows.microsoft.com/en-US/w...m-image-backup
>
>Another piece of trivia I learned in the last couple days. If the
>"Set up backup" button doesn't work (which you wouldn't be using
>in this case anyway), the command line equivalent of clicking
>that button, is
>
> sdclt.exe /configure
>
>A number of my backup buttons don't work right (thanks to the Acer
>added software...), but at least I've got a way to access that one.
>
>http://answers.microsoft.com/en-us/w...8-1b92a663f753
>
>HTH,
> Paul
>
>> On Mon, 19 Sep 2011 19:04:51 -0400, W8CCW <>
>> wrote:
>>
>>> Yesterday evening I discovered that I have not getting the backup that
>>> I want out of SyncToy. To facilitate working on the problem I decided
>>> to offload the current backup to a spare 1 TB drive in the same box. I
>>> created a folder called Drive E, did a control A select in Win 7
>>> Explorer, click on Copy, and select folder Drive, click on Paste and
>>> here we are 19 hours later waiting for 250 G to be transfered.
>>>
>>> Both drives are Seagate SATA on the same MSI MB. Is this normal?