Friday, July 14, 2017

Calculating HBase Tables disk space

If you want to get the size of the HBase tables,  the simple way is to use 'hadoop' command from your local hadoop setup.  The command is:

hadoop --config <your hbase-config-file location> fs -du  -h <folder where the data resides>

For example:
hadoop --config c:\hadoop\staging fs -du  -h /hbase/data/default/

This will display all the tables under that 'default' folder (Note: Based on your setup, the data folder may vary, but you get the idea).  The first column is the actual table size across the region servers.  The second column shows the total size including the replication.  The second column is important in deciding how much "total" space is required.

1.4 M    4.3 M    /hbase/data/default/Table1
79.9 M   239.8 M  /hbase/data/default/Table2
2.0 K    6.0 K    /hbase/data/default/Table3