GooseFS-FUSE can be mounted on the local file system of a Unix machine to access a GooseFS distributed file system. Using this feature, some standard command-line tools (such as ls, cat, and echo) can directly access the data in the GooseFS distributed file system. Additionally, more importantly, applications implemented in different languages, such as C, C++, Python, Ruby, Perl, and Java, can read and write GooseFS through standard POSIX interfaces (such as open, write, and read) without the need for any GooseFS client integration or settings.
GooseFS-FUSE is based on the FUSE project and supports most file system operations. However, due to the inherent attributes of GooseFS, such as its one-time and immutable file data model, the mounted file system is not fully consistent with the POSIX standard and still has certain limitations. Therefore, please first read the limitations to understand the role and limitations of this feature.
Installation Requirements
JDK 1.8 or higher
Linux system: libfuse 2.9.3 or higher (version 2.8.3 is available for use but will note some warnings)
After completing the configuration and starting the GooseFS cluster, launch a Shell on the node where GooseFS needs to be mounted, enter the $GOOSEFS_HOME directory, and execute the following commands:
$ integration/fuse/bin/goosefs-fuse mount mount_point [GooseFS_path]
This command will start a background Java process to mount the corresponding GooseFS path to the <mount_point> specified path. For example, the following command mounts the GooseFS path /people to the /mnt/people directory under the local file system.
$ integration/fuse/bin/goosefs-fuse mount /mnt/people /people
Starting goosefs-fuse on local host.
goosefs-fuse mounted at /mnt/people. See /lib/GooseFS/logs/fuse.log for logs
When GooseFS_path is not given, GooseFS-FUSE will default mount to the GooseFS root directory (/). You can call this command multiple times to mount GooseFS to different local directories. All GooseFS-FUSE instances share the $GOOSEFS_HOME\\logs\\fuse.log log file. This log file is helpful for fault troubleshooting.
Note:
The <mount_point> must be an empty folder in the local file system, and the user who starts the GooseFS-FUSE process must have read/write permissions for the mount point and its contents.
Uninstall GooseFS-FUSE
To uninstall GooseFS-FUSE, launch a Shell on the node and enter the $GOOSEFS_HOME directory to execute the following commands:
By default, if any read-write operations are ongoing, the unmount operation will wait for up to 120 seconds. If the read-write operations are still not completed after 120 seconds, the Fuse process will be forcibly ended, which can cause files being read or written to fail. You can add the -s parameter to avoid the Fuse process being forcibly ended. For example:
libexec: library file required for goosefs-fuse to run
goosefs-fuse-1.2.0: jar package for goosefs-fuse backend operation
log: log directory
Configuration Options
GooseFS-FUSE performs operations based on the standard GooseFS-core-client-fs. If you wish to use it like other applications' clients, you can customize the behavior of GooseFS-core-client-fs.
You can change client options by editing the $GOOSEFS_HOME/conf/goosefs-site.properties configuration file.
Note:
All changes should be completed before GooseFS-FUSE is started.
Limitations
Currently, GooseFS-FUSE supports most basic file system operations. However, due to some inherent characteristics of GooseFS, you need to pay attention to the following points:
A file can only be written sequentially once and cannot be modified. This means that to modify a file, you need to first delete it and then recreate it. For example, when the target file exists, the copy command cp will fail.
GooseFS has no concept of hard-link and soft-link, so it does not support related commands, such as ln. In addition, information about hard-link is not displayed in the output of ll.
Only when the GooseFS.security.group.mapping.class option of GooseFS is set to the value of ShellBasedUnixGroupsMapping, the user and group information of files corresponds to the user groups of the Unix system. Otherwise, the operations of chown and chgrp are not effective, and the user and group information returned by ll is the user and group information of the process that started GooseFS-FUSE.
Performance Considerations
Due to the combined use of FUSE and JNR, the performance of using a mounted file system is relatively poorer compared with directly using the native file system API.
Most performance issues are due to the fact that several copies exist in memory every time a read or write operation is performed, and FUSE sets the maximum granularity of write operations to 128KB. Its performance can be greatly improved by leveraging the FUSE write-back cache policy introduced in kernel 3.15 (but the libfuse 2.x userspace library currently does not support this feature).
GooseFS-FUSE Configuration Parameters
The following are GooseFS-FUSE related configuration parameters:
Parameter
Default Value
Description
goosefs.fuse.cached.paths.max
500
Define the size of the internal GooseFS-FUSE cache, which maintains the most commonly used conversions between local file system paths and GooseFS file paths.
goosefs.fuse.debug.enabled
false
Allow FUSE debug output, which will be redirected to the fuse.out log file in the goosefs.logs.dir specified directory.
goosefs.fuse.fs.name
goosefs-fuse
Descriptive name used for FUSE mounting file system.
goosefs.fuse.jnifuse.enabled
true
Use JNI-Fuse library for better performance. If disabled, JNR-Fuse will be used.
goosefs.fuse.shared.caching.reader.enabled
false
(Experimental) Use shared grpc data reader to achieve better performance in multi-process file reading via GooseFS JNI Fuse. Block data will be cached on the client, so the Fuse process requires more memory.
goosefs.fuse.logging.threshold
10s
Log FUSE API calls when query time exceeds the threshold.
goosefs.fuse.maxwrite.bytes
131072
FUSE write operation granularity (bytes). Note that currently 128KB is the upper bound of the Linux kernel limit.
goosefs.fuse.user.group.translation.enabled
false
Whether to convert GooseFS users and groups into corresponding Unix users and groups in the FUSE API. When set to false, all FUSE file users and groups will display as the user and group of the goosefs-fuse mounting thread.
FAQs
missing libfuse library file, need to install libfuse.
Method one
Installation command:
yum install fuse-devel
Check whether installed successfully:
find / -name libfuse.so*
Method two
Update the earlier version libfuse.so.2.9.2. The installation steps are as follows:
Note:
Install libfuse on CentOS 7. The default installation of CentOS 7 is libfuse.so.2.9.2.
First, download libfuse source code, and compile and generate libfuse.so.2.9.7.
tar -zxvf fuse-2.9.7.tar.gz
cd fuse-2.9.7/ && ./configure &&make&&makeinstall
echo -e '\\n/usr/local/lib'>> /etc/ld.so.conf
ldconfig
Secondly, after downloading, compiling, and generating libfuse.so.2.9.7, then follow the following steps to initiate installation and replacement.
1.1 execute the following commands, search for the library link of the old version libfuse.so.2.9.2
find / -name libfuse.so*
1.2 Execute the following commands, copy libfuse.so.2.9.7 to the location of the old version library libfuse.so.2.9.2.
cp /usr/local/lib/libfuse.so.2.9.7 /usr/lib64/
1.3 Execute the following commands, delete all links of the old version libfuse.so library.
rm -f /usr/lib64/libfuse.so
rm -f /usr/lib64/libfuse.so.2
1.4 execute the following commands, create a library link similar to the deleted old version link for libfuse.so.2.9.7.
