Re: Linking two files together

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Kevin J. Cummings wrote:
> On 05/26/2010 01:16 PM, Rector, David wrote:
>> Hello,
>>
>> I have studied various filesystems, and am fairly familiar with how they are structured. However, I am currently stuck on trying to do what seems like a simple thing.
>>
>> I would like to join two files together without having to physically copy bytes (i.e. I have vary large files, so I don't want to use 'cat'). It seems to me that it should be possible to simply modify the file entry in the filesystem such that the last inode of the first file points to the first inode of the second file. I guess this is similar to a "hard link", but used to join files rather than simply have another pointer to one file.
>>
>> I have seen 'mmv' and 'lxsplit' and they all seem to do the same thing, namely they want to physically copy the bytes in order to join two files together.
>>
>> Is there any such utility in linux to perform such a hard link to join or connect two files together without having to copy bytes?
> 
> If you could guarantee that the last extent used by the first file was
> completely full of data with no extraneous bytes, it might be possible
> to "merge" the extent maps of the 2 files into a single file entry.  If
> you cannot guarantee that, then you will have to copy bytes from the 2nd
> file to the end of the first file.

But everything becomes possible if the fileystem permits partially empty blocks
in the middle of the file. No filesystem does it AFAIK, but it is not a
big issue, as partial blocks (or compacted tails) are already permitted
at the end of the file. New filesystems use extents rather than blocks,
so if the extents are measured in bytes instead of 512b-blocks you can
just use a smaller extent in the middle of the file where the join happened.

At this point, you can support inplace-joining, inplace-inflating (add 10000 bytes
in this file at position 300000), inplace-erasure (remove 10000 bytes
at position 300000) and data shuffling (swap the first 50meg of the file with
the last 50meg).

With heavy usage you have just created a new kind of fragmentation, which can
be corrected with the usual defragmentation tools (including "cp").
(add that fragmentation is losing importance with the spreading of SSD)

Considering that sparse files have been a reality for decades and that
the implementation of operation with inside-file byte-grained extents
is not more difficult than truncate, I wonder if we will see something
of this kind in some advanced filesystem (btrfs?).

There are a lot of possible uses:
- delete/replace mail in mbox format repositories
- smart packaging (delete from tar, delete from zip)
- in-place iso creation
and.... just imagine.....
- video editing (!) add/remove/replace frames inside a 150GiB captured video

Where can you submit ideas to btrfs?
It also has COW, so everything becomes even more exciting...

-- 
   Roberto Ragusa    mail at robertoragusa.it
-- 
users mailing list
users@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines


[Index of Archives]     [Current Fedora Users]     [Fedora Desktop]     [Fedora SELinux]     [Yosemite News]     [Yosemite Photos]     [KDE Users]     [Fedora Tools]     [Fedora Docs]

  Powered by Linux