This command measure peformance of gfarm file system according to config file
.
config file
is written in YAML.
Example config files are (gfarm install directory)/share/gfarm/config/gfperf-[config|simple].yml .
Config file example is following.
database: {filename: "/var/www/database.db", check span: "10days",
backup: "/var/www/databse.db.bak" }
errlog: "/tmp/gfperf.err.log"
authentication: ["sharedsecret", "gsi_auth", "gsi"]
gfarm2fs_mountpoint: ["/mnt/gfarm2fs"]
metadata: [ {testdir: "gfarm:///tmp", number: "250"},
{testdir: "file:///mnt/gfarm2fs/tmp", number: "250"} ]
tree: [ {testdir: "gfarm:///tmp", width: "5", depth: "3"},
{testdir: "file:///mnt/gfarm2fs/tmp", width: "5", depth: "3"} ]
copy: [
{src: "file:///tmp", dst: "gfarm:///tmp", filesize: "1M",
bufsize: "4K", gfsd: "gfsd0.local"},
{src: "gfarm:///tmp", dst: "file:///tmp", filesize: "1M",
bufsize: "4K", gfsd: "gfsd0.local"},
{src: "file:///tmp", dst: "gfarm:///tmp", filesize: "1M",
bufsize: "4K", gfsd: "gfsd1.local"},
{src: "gfarm:///tmp", dst: "file:///tmp", filesize: "1M",
bufsize: "4K", gfsd: "gfsd1.local"},
{src: "file:///tmp", dst: "gfarm:///tmp", filesize: "1M",
bufsize: "4K", gfsd: "gfsd0.local", gfarm2fs: "/mnt/gfarm2fs"},
{src: "gfarm:///tmp", dst: "file:///tmp", filesize: "1M",
bufsize: "4K", gfsd: "gfsd0.local", gfarm2fs: "/mnt/gfarm2fs"},
{src: "file:///tmp", dst: "gfarm:///tmp", filesize: "1M",
bufsize: "4K", gfsd: "gfsd1.local", gfarm2fs: "/mnt/gfarm2fs"},
{src: "gfarm:///tmp", dst: "file:///tmp", filesize: "1M",
bufsize: "4K", gfsd: "gfsd1.local", gfarm2fs: "/mnt/gfarm2fs"}
]
read: [
{testdir: "gfarm:///tmp", filesize: "1M", bufsize: "4K",
gfsd: "gfsd0.local"},
{testdir: "gfarm:///tmp", filesize: "1M", bufsize: "4K",
gfsd: "gfsd1.local"},
{testdir: "gfarm:///tmp", random: "true", filesize: "1M",
bufsize: "4K", gfsd: "gfsd0.local"},
{testdir: "gfarm:///tmp", random: "true", filesize: "1M",
bufsize: "4K", gfsd: "gfsd1.local"},
{testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs", filesize: "1M",
bufsize: "4K", gfsd: "gfsd0.local"},
{testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs", filesize: "1M",
bufsize: "4K", gfsd: "gfsd1.local"},
{testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs", random: "true",
filesize: "1M", bufsize: "4K", gfsd: "gfsd0.local"},
{testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs", random: "true",
filesize: "1M", bufsize: "4K", gfsd: "gfsd1.local"}
]
write: [
{testdir: "gfarm:///tmp", filesize: "1M", bufsize: "4K",
gfsd: "gfsd0.local"},
{testdir: "gfarm:///tmp", filesize: "1M", bufsize: "4K",
gfsd: "gfsd1.local"},
{testdir: "gfarm:///tmp", random: "true", filesize: "1M",
bufsize: "4K", gfsd: "gfsd0.local"},
{testdir: "gfarm:///tmp", random: "true", filesize: "1M",
bufsize: "4K", gfsd: "gfsd1.local"},
{testdir: "gfarm:///tmp", overwrite: "true", filesize: "1M",
bufsize: "4K", gfsd: "gfsd0.local"},
{testdir: "gfarm:///tmp", overwrite: "true", filesize: "1M",
bufsize: "4K", gfsd: "gfsd1.local"},
{testdir: "gfarm:///tmp", overwrite: "true", random: "true",
filesize: "1M", bufsize: "4K", gfsd: "gfsd0.local"},
{testdir: "gfarm:///tmp", overwrite: "true", random: "true",
filesize: "1M", bufsize: "4K", gfsd: "gfsd1.local"},
{testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs", filesize: "1M",
bufsize: "4K", gfsd: "gfsd0.local"},
{testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs", filesize: "1M",
bufsize: "4K", gfsd: "gfsd1.local"},
{testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs", random: "true",
filesize: "1M", bufsize: "4K", gfsd: "gfsd0.local"},
{testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs", random: "true",
filesize: "1M", bufsize: "4K", gfsd: "gfsd1.local"},
{testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs",
overwrite: "true", filesize: "1M", bufsize: "4K",
gfsd: "gfsd0.local"},
{testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs",
overwrite: "true", filesize: "1M", bufsize: "4K",
gfsd: "gfsd1.local"},
{testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs",
overwrite: "true", random: "true", filesize: "1M", bufsize: "4K",
gfsd: "gfsd0.local"},
{testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs",
overwrite: "true", random: "true", filesize: "1M", bufsize: "4K",
gfsd: "gfsd1.local"}
]
replica: [
{testdir: "gfarm:///tmp", src: "gfsd0.local", dst: "gfsd1.local",
filesize: "1M"},
{testdir: "gfarm:///tmp", src: "gfsd1.local", dst: "gfsd0.local",
filesize: "1M"}
]
autoreplica: [
{testdir: "gfarm:///tmp", filesize: "1M", number: "3", replica: "1"},
{testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs", filesize: "1M",
number: "3", replica: "1"},
{testdir: "gfarm:///tmp", filesize: "1M", number: "3", replica: "2"},
{testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs", filesize: "1M",
number: "3", replica: "2"}
]
parallel: {
group1: [
{rhost: "gfsd0.local", type: "read", testdir: "gfarm:///tmp",
filesize: "1M", bufsize: "4K", gfsd: "gfsd0.local"},
{rhost: "gfsd0.local", type: "read", testdir: "gfarm:///tmp",
filesize: "1M", bufsize: "4K", gfsd: "gfsd1.local"}
],
group2: [
{rhost: "gfsd0.local", type: "write", testdir: "gfarm:///tmp",
filesize: "1M", bufsize: "4K", gfsd: "gfsd0.local"},
{rhost: "gfsd0.local", type: "write", testdir: "gfarm:///tmp",
filesize: "1M", bufsize: "4K", gfsd: "gfsd1.local"}
],
group3: [
{type: "replica", testdir: "gfarm:///tmp", src: "gfsd0.local",
dst: "gfsd1.local", filesize: "1M"},
{type: "replica", testdir: "gfarm:///tmp", src: "gfsd1.local",
dst: "gfsd0.local", filesize: "1M"}
]
}
parallel-autoreplica: {
group4: [
{testdir: "gfarm:///tmp1", filesize: "1M", number: "3",
replica: "2"},
{testdir: "gfarm:///tmp2", filesize: "1M", number: "3",
replica: "2"},
{testdir: "gfarm:///tmp3", filesize: "1M", number: "3",
replica: "2"},
{testdir: "gfarm:///tmp4", filesize: "1M", number: "3",
replica: "2"}
],
group5: [
{testdir: "gfarm:///tmp1", gfarm2fs: "/mnt/gfarm2fs",
filesize: "1M", number: "3", replica: "2"},
{testdir: "gfarm:///tmp2", gfarm2fs: "/mnt/gfarm2fs",
filesize: "1M", number: "3", replica: "2"}
]
}
Following sentences explain keywords of the YAML.
Specify databse filename in filename: . Specify backup of database file in backup: . check span: specifies time span of calculating average and standard deviation.
This commands prints error messsages in the file specified by errlog: . All error messages are also written in the database. If you don't need to error log file, this parameter can be omitted.
Specify authentication types of performance measuring. Authentication types are sharedsecret, gsi_auth and gsi. Performance test is executed on all specified types.
Specify gfarm2fs mount point. When start testing, this command mounts all gfarm2fs mount point automatically. At the end of testing, unmounted automatically. When changing authenticaion type, remounted automatically.
Perfomance measuring using gfperf-metadata. Parameters are gfperf-metadata long options.
Perfomance measuring using gfperf-tree. Parameters are gfperf-tree long options.
Perfomance measuring using gfperf-copy. Parameters are gfperf-copy long options.
Perfomance measuring using gfperf-read. Parameters are gfperf-read long options. If you need to test sequentially read, remove random option.
Perfomance measuring using gfperf-write. Parameters are gfperf-write long options. If you need to test sequentially write, remove random option. If you need to test append write, remove overwrite option.
Perfomance measuring using gfperf-replica. Parameters are gfperf-replica long options.
Perfomance measuring using gfperf-autoreplica. Parameters are gfperf-autoreplica long options.
Specify parallel test patterns. At first, specify group name that is a sinbol of parallel testing. Group name must be uniq in all parallel testing (includes parallel-autoreplica). Specify remote host in rhost: . This command uses ssh for executing command. type: specifies one of read, write, replica. These types corresponds to gfperf-parallel-read, gfperf-parallel-write, gfperf-parallel-replica.
Specify auto replication test in parallel. At first, specify group name that is a sinbol of parallel testing. Group name must be uniq in all parallel testing (includes parallel). Test parameters are gfperf-parallel-autoreplica long options. The result is average of gfperf-parallel-autoreplica outputs.