Name

gfperf.rb — Gfarm measure performance

Synopsis

gfperf.rb [-c, --check] config file

DESCRIPTION

This command measure peformance of gfarm file system according to config file. config file is written in YAML.

Example config files are (gfarm install directory)/share/gfarm/config/gfperf-[config|simple].yml .

OPTIONS

-c, --check

Check config file syntax.

EXAMPLES

Config file example is following.


database: {filename: "/var/www/database.db", check span: "10days",
   backup: "/var/www/databse.db.bak" }
errlog: "/tmp/gfperf.err.log"
authentication: ["sharedsecret", "gsi_auth", "gsi"]
gfarm2fs_mountpoint: ["/mnt/gfarm2fs"]
metadata: [ {testdir: "gfarm:///tmp", number: "250"},
            {testdir: "file:///mnt/gfarm2fs/tmp", number: "250"} ]
tree: [ {testdir: "gfarm:///tmp", width: "5", depth: "3"},
        {testdir: "file:///mnt/gfarm2fs/tmp", width: "5", depth: "3"} ]
copy: [
   {src: "file:///tmp", dst: "gfarm:///tmp", filesize: "1M",
    bufsize: "4K", gfsd: "gfsd0.local"},
   {src: "gfarm:///tmp", dst: "file:///tmp", filesize: "1M",
    bufsize: "4K", gfsd: "gfsd0.local"},
   {src: "file:///tmp", dst: "gfarm:///tmp", filesize: "1M",
    bufsize: "4K", gfsd: "gfsd1.local"},
   {src: "gfarm:///tmp", dst: "file:///tmp", filesize: "1M",
    bufsize: "4K", gfsd: "gfsd1.local"},
   {src: "file:///tmp", dst: "gfarm:///tmp", filesize: "1M",
    bufsize: "4K", gfsd: "gfsd0.local", gfarm2fs: "/mnt/gfarm2fs"},
   {src: "gfarm:///tmp", dst: "file:///tmp", filesize: "1M",
    bufsize: "4K", gfsd: "gfsd0.local", gfarm2fs: "/mnt/gfarm2fs"},
   {src: "file:///tmp", dst: "gfarm:///tmp", filesize: "1M",
    bufsize: "4K", gfsd: "gfsd1.local", gfarm2fs: "/mnt/gfarm2fs"},
   {src: "gfarm:///tmp", dst: "file:///tmp", filesize: "1M",
    bufsize: "4K", gfsd: "gfsd1.local", gfarm2fs: "/mnt/gfarm2fs"}
      ]
read: [
   {testdir: "gfarm:///tmp", filesize: "1M", bufsize: "4K",
    gfsd: "gfsd0.local"},
   {testdir: "gfarm:///tmp", filesize: "1M", bufsize: "4K",
     gfsd: "gfsd1.local"},
   {testdir: "gfarm:///tmp", random: "true", filesize: "1M",
    bufsize: "4K", gfsd: "gfsd0.local"},
   {testdir: "gfarm:///tmp", random: "true", filesize: "1M",
    bufsize: "4K", gfsd: "gfsd1.local"},
   {testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs", filesize: "1M",
    bufsize: "4K", gfsd: "gfsd0.local"},
   {testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs", filesize: "1M",
    bufsize: "4K", gfsd: "gfsd1.local"},
   {testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs", random: "true",
    filesize: "1M", bufsize: "4K", gfsd: "gfsd0.local"},
   {testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs", random: "true",
    filesize: "1M", bufsize: "4K", gfsd: "gfsd1.local"}
      ]
write: [
   {testdir: "gfarm:///tmp", filesize: "1M", bufsize: "4K",
    gfsd: "gfsd0.local"},
   {testdir: "gfarm:///tmp", filesize: "1M", bufsize: "4K",
    gfsd: "gfsd1.local"},
   {testdir: "gfarm:///tmp", random: "true", filesize: "1M",
    bufsize: "4K", gfsd: "gfsd0.local"},
   {testdir: "gfarm:///tmp", random: "true", filesize: "1M",
    bufsize: "4K", gfsd: "gfsd1.local"},
   {testdir: "gfarm:///tmp", overwrite: "true", filesize: "1M",
    bufsize: "4K", gfsd: "gfsd0.local"},
   {testdir: "gfarm:///tmp", overwrite: "true", filesize: "1M",
    bufsize: "4K", gfsd: "gfsd1.local"},
   {testdir: "gfarm:///tmp", overwrite: "true", random: "true",
    filesize: "1M", bufsize: "4K", gfsd: "gfsd0.local"},
   {testdir: "gfarm:///tmp", overwrite: "true", random: "true",
    filesize: "1M", bufsize: "4K", gfsd: "gfsd1.local"},
   {testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs", filesize: "1M",
    bufsize: "4K", gfsd: "gfsd0.local"},
   {testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs", filesize: "1M",
    bufsize: "4K", gfsd: "gfsd1.local"},
   {testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs", random: "true",
    filesize: "1M", bufsize: "4K", gfsd: "gfsd0.local"},
   {testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs", random: "true",
    filesize: "1M", bufsize: "4K", gfsd: "gfsd1.local"},
   {testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs",
    overwrite: "true", filesize: "1M", bufsize: "4K",
    gfsd: "gfsd0.local"},
   {testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs",
    overwrite: "true", filesize: "1M", bufsize: "4K",
    gfsd: "gfsd1.local"},
   {testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs",
    overwrite: "true", random: "true", filesize: "1M", bufsize: "4K",
    gfsd: "gfsd0.local"},
   {testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs",
    overwrite: "true", random: "true", filesize: "1M", bufsize: "4K",
    gfsd: "gfsd1.local"}
      ]
replica: [
   {testdir: "gfarm:///tmp", src: "gfsd0.local", dst: "gfsd1.local",
    filesize: "1M"},
   {testdir: "gfarm:///tmp", src: "gfsd1.local", dst: "gfsd0.local",
    filesize: "1M"}
 ]
autoreplica: [
   {testdir: "gfarm:///tmp", filesize: "1M", number: "3", replica: "1"},
   {testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs", filesize: "1M",
    number: "3", replica: "1"},
   {testdir: "gfarm:///tmp", filesize: "1M", number: "3", replica: "2"},
   {testdir: "gfarm:///tmp", gfarm2fs: "/mnt/gfarm2fs", filesize: "1M",
    number: "3", replica: "2"}
 ]
parallel: {
  group1: [
   {rhost: "gfsd0.local", type: "read", testdir: "gfarm:///tmp",
    filesize: "1M", bufsize: "4K", gfsd: "gfsd0.local"},
   {rhost: "gfsd0.local", type: "read", testdir: "gfarm:///tmp",
    filesize: "1M", bufsize: "4K", gfsd: "gfsd1.local"}
        ],
  group2: [
   {rhost: "gfsd0.local", type: "write", testdir: "gfarm:///tmp",
    filesize: "1M", bufsize: "4K", gfsd: "gfsd0.local"},
   {rhost: "gfsd0.local", type: "write", testdir: "gfarm:///tmp",
    filesize: "1M", bufsize: "4K", gfsd: "gfsd1.local"}
       ],
  group3: [
   {type: "replica", testdir: "gfarm:///tmp", src: "gfsd0.local",
    dst: "gfsd1.local", filesize: "1M"},
   {type: "replica", testdir: "gfarm:///tmp", src: "gfsd1.local",
    dst: "gfsd0.local", filesize: "1M"}
       ]
}
parallel-autoreplica: {
  group4: [
     {testdir: "gfarm:///tmp1", filesize: "1M", number: "3",
      replica: "2"},
     {testdir: "gfarm:///tmp2", filesize: "1M", number: "3",
      replica: "2"},
     {testdir: "gfarm:///tmp3", filesize: "1M", number: "3",
      replica: "2"},
     {testdir: "gfarm:///tmp4", filesize: "1M", number: "3",
      replica: "2"}
       ],
  group5: [
     {testdir: "gfarm:///tmp1", gfarm2fs: "/mnt/gfarm2fs",
      filesize: "1M", number: "3", replica: "2"},
     {testdir: "gfarm:///tmp2", gfarm2fs: "/mnt/gfarm2fs",
      filesize: "1M", number: "3", replica: "2"}
       ]    
}

Following sentences explain keywords of the YAML.

database

Specify databse filename in filename: . Specify backup of database file in backup: . check span: specifies time span of calculating average and standard deviation.

errlog

This commands prints error messsages in the file specified by errlog: . All error messages are also written in the database. If you don't need to error log file, this parameter can be omitted.

authentication

Specify authentication types of performance measuring. Authentication types are sharedsecret, gsi_auth and gsi. Performance test is executed on all specified types.

gfarm2fs_mountpoint

Specify gfarm2fs mount point. When start testing, this command mounts all gfarm2fs mount point automatically. At the end of testing, unmounted automatically. When changing authenticaion type, remounted automatically.

metadata

Perfomance measuring using gfperf-metadata. Parameters are gfperf-metadata long options.

tree

Perfomance measuring using gfperf-tree. Parameters are gfperf-tree long options.

copy

Perfomance measuring using gfperf-copy. Parameters are gfperf-copy long options.

read

Perfomance measuring using gfperf-read. Parameters are gfperf-read long options. If you need to test sequentially read, remove random option.

write

Perfomance measuring using gfperf-write. Parameters are gfperf-write long options. If you need to test sequentially write, remove random option. If you need to test append write, remove overwrite option.

replica

Perfomance measuring using gfperf-replica. Parameters are gfperf-replica long options.

autoreplica

Perfomance measuring using gfperf-autoreplica. Parameters are gfperf-autoreplica long options.

parallel

Specify parallel test patterns. At first, specify group name that is a sinbol of parallel testing. Group name must be uniq in all parallel testing (includes parallel-autoreplica). Specify remote host in rhost: . This command uses ssh for executing command. type: specifies one of read, write, replica. These types corresponds to gfperf-parallel-read, gfperf-parallel-write, gfperf-parallel-replica.

parallel-autoreplica

Specify auto replication test in parallel. At first, specify group name that is a sinbol of parallel testing. Group name must be uniq in all parallel testing (includes parallel). Test parameters are gfperf-parallel-autoreplica long options. The result is average of gfperf-parallel-autoreplica outputs.

SEE ALSO

gfperf-metadata(1), gfperf-tree(1), gfperf-copy(1), gfperf-read(1), gfperf-write(1), gfperf-replica(1), gfperf-autoreplica(1), gfperf-parallel-read(1), gfperf-parallel-write(1), gfperf-parallel-replica(1), gfperf-parallel-autoreplica(1),

NOTES

Using ssh command execution, no password is recommended.

Bash doesn't read ~/.bashrc in some envronment. Please check before running gfperf.rb.