[NBLUG/talk] Synchronizing strategies
Omar Eljumaily
omar at omnicode.com
Sat Nov 22 11:50:55 PST 2014
I think this is what I'm looking for.
http://manpages.ubuntu.com/manpages/trusty/man7/inotify.7.html
Thanks Zack and Jordan. I think some combination of rsync, srcipts, and
C++ code is what I'm going to use. I mainly wanted to know if there was
something like "just install this and it will solve your problem," which
there seems not to be.
I guess the data size is not so much an issue as the number of files I'm
talking about synching which I think is about 3 or 4 million files. I
know that Hostgator limits the number of files for inexpensive accounts
to about 200k, although they claim unlimited data size (or used to).
Thanks,
Omar
On 11/22/2014 11:18 AM, Omar Eljumaily wrote:
> I think it would be cheaper to just get directory/file mod listings,
> which I'm already paranoid would create undo overhead. I know that
> Samba plugins can tell you when files have been modified without
> searching. I'm wondering if there's such a thing for Linux file systems.
>
> Thanks,
>
> Omar
>
>
> On 11/22/2014 11:14 AM, Zack Gold wrote:
>>
>> You could write a clever script that stores md5sums of every file and
>> check I'd it's changed.
>>
>> On Nov 22, 2014 2:13 PM, "Omar Eljumaily" <omar at omnicode.com
>> <mailto:omar at omnicode.com>> wrote:
>>
>> I'd like to have the knowledge of when a file changes. I don't
>> know how to get those notifications in linux. Does anybody know?
>>
>> Also, not all files get closed to the point where they can be
>> copied every time they're modified. Windows clients are
>> notorious for keeping files open all day long without letting
>> other processes copy them.
>>
>> The thing with the could side is that I don't want to be the
>> person responsible for running the cloud server, so I'm going to
>> have to work within the cloud provider's framework. The ones I'm
>> looking at right now are Google and 1and1.com <http://1and1.com>
>> (which seems to be pretty cheap, about $10/month for 1 terrabyte).
>>
>> Thanks,
>>
>> Omar
>>
>>
>> On 11/22/2014 10:57 AM, Jordan Erickson wrote:
>>
>> I use rsync+ssh+rsnapshot+cron for a (albeit daily) backup of
>> multiple
>> VMs in "the cloud" (my cloud) totaling about 1.5TB total to a
>> local
>> disk, then another rsnapshot to an external disk about every
>> month. It's
>> fast. rsync is designed to take large amounts of data and
>> sync them
>> efficiently.
>>
>> Is there a requirement for "every hour" (and not something
>> more frequent
>> like, every time a file changes)? If not, maybe a trigger
>> that whenever
>> a file changes, it's sync'd... I'm sure multiple avenues are
>> possible
>> with some simple script-fu (and probably plenty of
>> ready-to-go packages
>> as well).
>>
>>
>> Jordan Erickson (PGP: 0x78DD41CB)
>> LNS: 707-636-5678 <tel:707-636-5678>,
>> http://logicalnetworking.net
>>
>>
>> On 11/22/2014 10:50 AM, steve wrote:
>>
>> On 11/22/2014 10:43 AM, Zack Gold wrote:
>>
>> rsync + cron?
>>
>> On Nov 22, 2014 1:36 PM, "Omar Eljumaily"
>> <omar at omnicode.com <mailto:omar at omnicode.com>
>> <mailto:omar at omnicode.com
>> <mailto:omar at omnicode.com>>> wrote:
>>
>> Does anybody know about file synchronization
>> strategies, mainly for
>> synching from a local file server to a cloud?
>> What I'm trying to
>> accomplish is:
>>
>> For a terebyte???
>> _______________________________________________
>> talk mailing list
>> talk at nblug.org <mailto:talk at nblug.org>
>> http://nblug.org/cgi-bin/mailman/listinfo/talk
>>
>> _______________________________________________
>> talk mailing list
>> talk at nblug.org <mailto:talk at nblug.org>
>> http://nblug.org/cgi-bin/mailman/listinfo/talk
>>
>>
>> _______________________________________________
>> talk mailing list
>> talk at nblug.org <mailto:talk at nblug.org>
>> http://nblug.org/cgi-bin/mailman/listinfo/talk
>>
>>
>>
>> _______________________________________________
>> talk mailing list
>> talk at nblug.org
>> http://nblug.org/cgi-bin/mailman/listinfo/talk
>
>
>
> _______________________________________________
> talk mailing list
> talk at nblug.org
> http://nblug.org/cgi-bin/mailman/listinfo/talk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://nblug.org/pipermail/talk/attachments/20141122/f0eebd92/attachment.html>
More information about the talk
mailing list