Quantcast

Maximum PC

It is currently Wed Oct 01, 2014 1:17 pm

All times are UTC - 8 hours




Post new topic Reply to topic  [ 6 posts ] 
Author Message
 Post subject: File duplication and favorites libraries management?
PostPosted: Sat Oct 19, 2013 10:29 am 
Klamath
Klamath
User avatar

Joined: Fri Jun 16, 2006 12:36 pm
Posts: 292
Management of the original files and their duplicates is sloppy on the drive--there are more than 600 duplicate files. The drive has the original files in their original folder, but multiple users of the drive have copied files into "favorites" folders. The result is that these favorites folders have files copied from dozens of original file folders. Changes to original files (like tags, renaming, or image edits) do not update their duplicate files.

Normally when I'm working on a project, I'm not embedding the files in the project--they are asset references only. When I'm creating a folder of favorites, I'll make it a subfolder in the original folder and cut the files from the original folder--this keeps the files in the original folder and collects the favorited files in the same location. This can't be done when the favorites are from multiple folders--the same folder can't be a subfolder of multiple folders.

The goal is to 1. Eliminate duplicate files, 2. Keep the pre-existing lists of favorite files in a non-duplicating format. I see at least two ways of doing this: libraries and shortcuts. It should be done with libraries, because shortcuts do not let you preview them all at once--you have to click each link, view it, close it, and click the next link. With a library, you can see all of the images in one window, and can view the images in full screen in sequence by hitting the left and right arrow keys. To comply with goal 2., the library must be in the same folder as the old folder that contained the duplicate files, so I need to know how libraries can be cut pasted into a subfolder. EDIT: I also need to know how libraries can include individual images instead of entire folders.

I need help writing a script that will let me automate this. This is a basic idea of how the script would work. I'm using SearchMyFiles to find the duplicate files. Output of a list of duplicates is not supported by SearchMyFiles, so if I continue using it to find the duplicates, it will require editing the source code. This is actually what I'm manually doing, but it is extremely slow going.
Code:
//1 search big folder for all duplicates: C:\Users\[username]\Documents\750GB drive\;
//2 list duplicate files;
//3 create library in C:\Users\[username]\Documents\750GB drive\DummyUser's favorites\;
//4 select files from C:\Users\[username]\Documents\750GB drive\original file\;
//5 add original files from C:\Users\[username]\Documents\750GB drive\original file\ to library in C:\Users\[username]\Documents\750GB drive\DummyUser's favorites\;
//6 delete duplicate files from C:\Users\[username]\Documents\750GB drive\DummyUser's favorites\;
//7 do steps 3-6 while there are file duplicates remaining;


Top
  Profile  
 
 Post subject: Re: File duplication and favorites libraries management?
PostPosted: Sat Oct 19, 2013 1:48 pm 
Smithfield
Smithfield

Joined: Sun Jun 18, 2006 7:37 pm
Posts: 5300
Here's the best way to tackle this problem, because this is a clunky way of doing shared projects.

1. Merge everything back or however you want to do this so that you have the most up-to-date single source of truth.
2. Create a subversion server and use that. The problem with what I see here you're simply copying everything back of whatever you think is the most up-to-date regardless of things in "obsolete" versions that may be useful. And you won't have any sort of way to roll back changes. An SVN server will allow you to track all changes, branch projects, merge files back in needed, and roll back changes.


Top
  Profile  
 
 Post subject: Re: File duplication and favorites libraries management?
PostPosted: Sat Oct 19, 2013 2:44 pm 
Klamath
Klamath
User avatar

Joined: Fri Jun 16, 2006 12:36 pm
Posts: 292
I gave up using SearchMyFiles and tried Duplicate Cleaner Free, which didn't allow making hard links in the free version, so I gave up on that and tried CloneSpy. I set it to create hard links, and manually went through the 600 duplicate groups.

The problem is now that CloneSpy detects those same hard linked files as duplicate files, which means that any time a new duplicate file from another drive or folder is scanned against the original one, I'm going to have to go through the scan all over again. I need to figure out how to tell CloneSpy to ignore hard links when scanning a folder that has already been consolidated. I can create a checksum for new folders imported to the drive, but once they're on the drive, I'd have to rescan all of the hard linked items to make sure the original one is still where it is supposed to be.

I might try SVN, looks like VisualSVN plus TortoiseSVN (Although I'm sure as soon as I say that, someone else is going to come in to the thread with their own pet program that is absolutely superior in every way to the previously mentioned ones without explaining that it requires a background in nonconforming program abandonment management in LISP heuristics. Any time tutorials are involved, I start to get wary because I know lying ahead is an issue that requires additional troubleshooting a bug and asking questions that might never be resolved). Might be the only choice if I can't get CloneSpy to do what it needs to do.


Top
  Profile  
 
 Post subject: Re: File duplication and favorites libraries management?
PostPosted: Sat Oct 19, 2013 4:14 pm 
Smithfield
Smithfield

Joined: Sun Jun 18, 2006 7:37 pm
Posts: 5300
If you're doing a project that's being developed by multiple people and you want to have a handle on versions, file updates, and "one source of truth", then subversion or similar is pretty much the only way to go. Anything else might be an exercise in frustration and futility.


Top
  Profile  
 
 Post subject: Re: File duplication and favorites libraries management?
PostPosted: Sat Feb 08, 2014 12:19 am 
8086
8086

Joined: Fri Feb 07, 2014 10:26 pm
Posts: 2
Duplication creates saving problem and it slow down the working of the computer / site; that is irritating and the user loses his interest. DuplicateFilesDeleter is the best solution of this problem. It is the best duplicate finder and deleter. It gives you the option of automatic deleting of all the duplicate files.


Top
  Profile  
 
 Post subject: Re: File duplication and favorites libraries management?
PostPosted: Sun Feb 09, 2014 10:21 pm 
8086
8086

Joined: Fri Feb 07, 2014 10:26 pm
Posts: 2
I think Duplicate Files Deleter software is the best solution of dupilication problem. You can easily delete duplicate files by using this software from any location of your dick. It works very fast .This program is multi-threaded and performs scans quickly.


Top
  Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 6 posts ] 

All times are UTC - 8 hours


Who is online

Users browsing this forum: No registered users and 0 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group

© 2014 Future US, Inc. All rights reserved.