Hadoop dashboard

Now we have dashboard for out hadoop clusters. On github you can see sourcecode (link is bottom).

At sourcecode are (for Hadoop) the most important jobs hdfs_status.rb, yarn_status.rb, hbase_status.rb.

  • hdfs_status.rb - get count live, dead and decommission nodes
  • yarn_status.rb - get count running, lost and unhealth nodemanagers and running/failing jobs (more info bottom)
  • hbase_status.rb - get count live regionservers, dead regionservers and region in transitions

Here you can see cluster with HDFS + YARN + HBase.

dashboard

In next screenshot you can see failing a running jobs. This jobs are download and parse from http://$masterurl:8088/ws/v1/cluster/apps


Failing jobs shows:

  • name
  • datetime when job failed
  • last success complete job with same name

Running jobs shows:

  • name
  • datetime start
  • owner
  • type (yarn, spark, mr, ...)

dashboard

Here is sourcecode for Hadoop dashboard. Dashboard use project called dashing (for more info check out github).