Ubuntu-16.04 [SOLVED]: Understanding ubuntu ZFS compression status variables

Ubuntu-16.04 [SOLVED]: Understanding ubuntu ZFS compression status variables

Home Forums Ubuntu 16.04 Ubuntu-16.04 [SOLVED]: Understanding ubuntu ZFS compression status variables

Tagged: ,

Viewing 1 post (of 1 total)
  • Author
    Posts
  • #36991

    Anonymous

    QuestionQuestion

    I have a Ubuntu VM which has a volume containing some very, very compressible datasets on it.

    For this reason, I’ve converted the volume in question to a ZFS volume, because I can then use ZFS compression.

    This is all working really well, but I’m confused by some of the ZFS status output.

    durr@graphical:/tank$ du . -h --max-depth=1; echo -----; du . -h --apparent-size --max-depth=1
    1.9G    ./carbon
    1.9G    .
    -----
    193G    ./carbon
    193G    .
    

    Note: /tank/ is the mountpoint of the ZFS volume.

    So, give the above, I’m currently getting ~1% compression ratio (this is expected, the volume is almost entirely Carbon data files, which are mostly empty, so they should be extremely compressible).

    However, if I ask ZFS about the volume:

    durr@graphical:/tank$ sudo zfs get all tank
    NAME  PROPERTY              VALUE                  SOURCE
    tank  type                  filesystem             -
    tank  creation              Mon Dec 25  7:27 2017  -
    tank  used                  1.87G                  -
    tank  available             239G                   -
    tank  referenced            1.85G                  -
    tank  compressratio         4.39x                  -
    tank  mounted               yes                    -
    tank  quota                 none                   default
    tank  reservation           none                   default
    tank  recordsize            128K                   default
    tank  mountpoint            /tank                  default
    tank  sharenfs              off                    default
    tank  checksum              on                     default
    tank  compression           on                     local
    tank  atime                 on                     default
    tank  devices               on                     default
    tank  exec                  on                     default
    tank  setuid                on                     default
    tank  readonly              off                    default
    tank  zoned                 off                    default
    tank  snapdir               hidden                 default
    tank  aclinherit            restricted             default
    tank  canmount              on                     default
    tank  xattr                 on                     default
    tank  copies                1                      default
    tank  version               5                      -
    tank  utf8only              off                    -
    tank  normalization         none                   -
    tank  casesensitivity       sensitive              -
    tank  vscan                 off                    default
    tank  nbmand                off                    default
    tank  sharesmb              off                    default
    tank  refquota              none                   default
    tank  refreservation        none                   default
    tank  primarycache          all                    default
    tank  secondarycache        all                    default
    tank  usedbysnapshots       0                      -
    tank  usedbydataset         1.85G                  -
    tank  usedbychildren        18.7M                  -
    tank  usedbyrefreservation  0                      -
    tank  logbias               latency                default
    tank  dedup                 on                     local
    tank  mlslabel              none                   default
    tank  sync                  standard               default
    tank  refcompressratio      4.40x                  -
    tank  written               1.85G                  -
    tank  logicalused           8.13G                  -
    tank  logicalreferenced     8.13G                  -
    tank  filesystem_limit      none                   default
    tank  snapshot_limit        none                   default
    tank  filesystem_count      none                   default
    tank  snapshot_count        none                   default
    tank  snapdev               hidden                 default
    tank  acltype               off                    default
    tank  context               none                   default
    tank  fscontext             none                   default
    tank  defcontext            none                   default
    tank  rootcontext           none                   default
    tank  relatime              on                     temporary
    tank  redundant_metadata    all                    default
    tank  overlay               off                    default
    

    ZFS is reporting a compression ratio of either 4.39x or 4.40x, depending on where you look. However, with the ~1% compression ratio from earlier, I’d expect to see either 0.01x or 99.0x, depending on how ZFS represents it’s status.

    Googling around, I can’t seem to find the documentation on the compressratio member. It definitely changes as you move data around, because I’ve seen it vary, but what is it actually telling me?


    Thinking about it, I also have ZFS deduplication turned on for this volume, so I thought it could be deduplicating the empty blocks. However, that doesn’t seem correct:

    durr@graphical:/tank$ sudo zpool get all tank
    NAME  PROPERTY                    VALUE                       SOURCE
    tank  size                        248G                        -
    tank  capacity                    0%                          -
    tank  altroot                     -                           default
    tank  health                      ONLINE                      -
    tank  guid                        11995166271724776732        default
    tank  version                     -                           default
    tank  bootfs                      -                           default
    tank  delegation                  on                          default
    tank  autoreplace                 off                         default
    tank  cachefile                   -                           default
    tank  failmode                    wait                        default
    tank  listsnapshots               off                         default
    tank  autoexpand                  off                         default
    tank  dedupditto                  0                           default
    tank  dedupratio                  1.12x                       -
    tank  free                        246G                        -
    tank  allocated                   1.69G                       -
    tank  readonly                    off                         -
    tank  ashift                      0                           default
    tank  comment                     -                           default
    tank  expandsize                  -                           -
    tank  freeing                     0                           default
    tank  fragmentation               1%                          -
    tank  leaked                      0                           default
    tank  feature@async_destroy       enabled                     local
    tank  feature@empty_bpobj         enabled                     local
    tank  feature@lz4_compress        active                      local
    tank  feature@spacemap_histogram  active                      local
    tank  feature@enabled_txg         active                      local
    tank  feature@hole_birth          active                      local
    tank  feature@extensible_dataset  enabled                     local
    tank  feature@embedded_data       active                      local
    tank  feature@bookmarks           enabled                     local
    tank  feature@filesystem_limits   enabled                     local
    tank  feature@large_blocks        enabled                     local
    

    I have no idea where the extra data is, from ZFS’s perspective. I think the files are sparse. Does ZFS not dedicate the disk-space to sparse files immediately?

Viewing 1 post (of 1 total)

You must be logged in to reply to this topic.