Created:        2022-11-07 Mon
Last modified:  2022-11-17 Thu

Hadoop: HDFS Quotas

HDFS quota commands are well explained in this post. The only drawback is old Hadoop version: 0.20.

Here are commands for Haddop 3.3.4.

• Get file/dir sizes, quotas

  • hdfs dfs -count -q -h -v  <hdfs_path>

  • hdfs dfs -count -q -v <hdfs_path>

• Health report (inculding size and replication factor)

  • hdfs fsck <hdfs_path>

• Setting replication factor

  • hdfs dfs -setrep -w <num_rep> <hdfs_path>

• Print requested information (can be useful in scripts)

  • hdfs dfs -stat [format] <hdfs_path>

  • [format] (%y is used by default)

    • permissions

      • octal (%a)

      • symbolic (%A)

    • filesize in bytes (%b)

    • type (%F)

    • group name of owner (%g)

    • name (%n)

    • block size (%o)

    • replication (%r)

    • user name of owner(%u)

    • access date(%x, %X)

    • modification date (%y, %Y)

      • %x and %y show UTC date as "yyyy-MM-dd HH:mm:ss"

      • %X and %Y show milliseconds since January 1, 1970 UTC