Archive for category Ganglia

How to feed data into Graphite?

Posted by Maz in Ganglia, Graphite, Monitoring Tools on April 17, 2013

I came across of this when wanting to back fill the data from RRDs to Graphite/whisper… The following link explains how to use nc to insert data.

http://graphite.readthedocs.org/en/1.0/feeding-carbon.html

I simply use nc in my shell script to feed it to Carbon. Below is part of the script that does that.

CARBON_PORT="2003"
CARBON_SERVER="host1"
CARBON_PREFIX="ganglia."
GRAPHITE_PATH="${CARBON_PREFIX}${CLUSTER_GRAPHITE}.${SERVER_GRAPHITE}.${METRIC_GRAPHITE}"
echo "${GRAPHITE_PATH} ${VALUE} ${TIMESTAMP}" | nc ${CARBON_SERVER} ${CARBON_PORT}

No Comments

How to integrate Ganglia 3.3.x+ and Graphite 0.9.x?

Posted by Maz in Ganglia on April 16, 2013

As of Ganglia 3.3.x the Graphite Integration plugin was integrated into gmetad. So if configured properly, when gmetad gathers the data it writes to RRDs and sends the metrics to Graphite.

Now, you might ask why would you want to integrate with Graphite?
Better graphs, more calculation options such Standard Deviation, Moving Average, many other tools that integrate to Graphite such as Tattle or Dynamic Graphing tools and much much more are reasons why you would want to integrate with Graphite.

Why would you not simply go to Graphite and skip Ganglia?
Well, in our case we needed Ganglia because we’re using HBase and Cloudera and unfortunately Cloudera provides integration to Ganglia and I believe JMX.. But if you have the option of going directly to Graphite then go for it. The Graphite website provide pretty good instructions for the install..

Finally, to integrate Ganglia 3.3.x+ with Graphite simply add the following lines to your gmetad.conf file. 
carbon_server  server_name/ip
carbon_port 2003
graphite_prefix "ganglia"
  • Carbon_Port is not mandatory and can be omitted. The default port for Carbon is 2003 so if you haven’t changed it in Graphite settings then you can skip it.
  • Graphite_prefix is the name that all your clusters and metrics will reside under. Its just a directory structure.
  • You must restart gmetad service for changes to take effect. 
    service gmetad restart
  • Once you restart the service you must monitor /var/log/messages for any errors.

No Comments

Installing Ganglia 3.5.x

Posted by Maz in Ganglia, Monitoring Tools on April 16, 2013

Notes:

  • The instructions were created using CentOS 6.2 OS
  • Ganglia is made up of few components: gmetad, gmond and web UI
  • The Server includes all 3 components but the Client only includes gmond
  • These steps are for both Server and Clients. At times the instructions splits to Server and Client. If a step doesn’t include Server or Client then you can assume that the steps is for both Server and Client install.

Pre-install Steps:
  1. Selinux: Either disable it or make appropriate changes to allow the install to go through and the application will be able to listen on appropriate ports.To stop selinux permanently modify 
    /etc/sysconfig/selinux
    and change 
    selinux
    to 
    disabled
    .
  2. IPTables: Same as Selinux. Disable it or allow ports through.Use the following commands to disable IPTables: 
    service iptables stop;
    chkconfig iptables off;
    chkconfig --list iptables;
    iptables -F
    service iptables save

Installing Dependencies:

  • Server Components
    • Use Yum to install the following packages:

      • httpd.x86_64
      • httpd-devel.x86_64
      • php.x86_64
      • php-cli.x86_64
      • php-devel.x86_64
      • php-gd.x86_64
      • php-mysql.x86_64
      • php-pear.noarch
      • rrdtool-php.x86_64
      • rrdtool-devel.x86_64
      • rrdtool.x86_64
      • rrdtool-perl.x86_64
      • glibc.x86_64
      • glib2-devel.x86_64
      • glibc-devel.x86_64
      • glib2.x86_64
      • glibc-utils.x86_64
      • glibc-common.x86_64
      • gcc.x86_64
      • gcc-c++.x86_64
      • libgcc.x86_64
      • compat-gcc-34.x86_64
      • pcre.x86_64
      • pcre-devel.x86_64
      • python.x86_64
      • rrdtool-python.x86_64
      • yum install httpd.x86_64 httpd-devel.x86_64 php.x86_64 php-cli.x86_64 php-devel.x86_64 php-gd.x86_64 php-mysql.x86_64 php-pear.noarch rrdtool-php.x86_64 rrdtool-devel.x86_64 rrdtool.x86_64 rrdtool-perl.x86_64 glibc.x86_64 glib2-devel.x86_64 glibc-devel.x86_64 glib2.x86_64 glibc-utils.x86_64 glibc-common.x86_64 gcc.x86_64 gcc-c++.x86_64 libgcc.x86_64 compat-gcc-34.x86_64 pcre.x86_64 pcre-devel.x86_64 python.x86_64 rrdtool-python.x86_64
  • Client Components
    • Use Yum to install the following packages:
      • glibc.x86_64
      • glib2-devel.x86_64
      • glibc-devel.x86_64
      • glib2.x86_64
      • glibc-utils.x86_64
      • gcc-c++.x86_64
      • compat-gcc-34.x86_64
      • pcre.x86_64
      • pcre-devel.x86_64
      • python.x86_64
      • expat.x86_64
      • expat-devel.x86_64
      • apr.x86_64
      • apr-util.x86_64
      • apr-devel.x86_64
      • apr-util-devel.x86_64
      • gcc.x86_64
      • libgcc.x86_64
      • gcc-objc.x86_64
      • yum install glibc.x86_64 glib2-devel.x86_64 glibc-devel.x86_64 glib2.x86_64 glibc-utils.x86_64 gcc-c++.x86_64 compat-gcc-34.x86_64 pcre.x86_64 pcre-devel.x86_64 python.x86_64 expat.x86_64 expat-devel.x86_64 apr.x86_64 apr-util.x86_64 apr-devel.x86_64 apr-util-devel.x86_64 gcc.x86_64 libgcc.x86_64 gcc-objc.x86_64
Install Ganglia (gmetad/gmond):
  1. Install libconfuse RPMS: 
    rpm --ivh libconfuse--2.6-2.el6.rf.x86_64.rpm libconfuse--devel--2.6-2.el6.rf.x86_64.rpm
  2. Decompress Ganglia gz file to /opt directory: 
    tar --xzvf ganglia-3.5.0.tar.gz -C /opt
  3. Change directory to /opt and rename ganglia-3.5.0 to ganglia: 
    cd /opt;
mv ganglia-3.5.0 ganglia;
cd ganglia;
  4. Run Configure command:
    • Server (with gmetad): 
      ./configure --with-libpcre=no --with-gmetad
    • Client: 
      ./configure --with-libpcre=no
  5. Run make and make install 
    make
    make install
  6. (Server) Create RRD directory. Note that we run ganglia as nobody so here we change the ownership to nobody user. 
    mkdir -p /var/lib/ganglia/rrds;chown -R nobody /var/lib/ganglia/rrds;
Configure GMOND:
  1. Create /etc/ganglia directory 
    mkdir -p /etc/ganglia
  2. Create default configuration. 
    gmond --default_config > /etc/ganglia/gmond.conf
  3. Create symbolic link from default location ganglia looks for configuration to /etc/ganglia. This is because its easier to manage all configuration files in the same place. 
    ln -s /etc/ganglia/gmond.conf /usr/local/etc/gmond.conf;
  4. Change 
    /etc/ganglia/gmond.conf
    to have desired metrics and port number. Note that each cluster/group must have its own unique port number. The default is 8649.
  5. Change the init file to point to correct binary: 
    vi /opt/ganglia/gmond/gmond.init
    change 
    GMOND
    to 
    /usr/local/sbin/gmond
    .
  6. Copy gmond.init to the init.d directory and add to start up: 
    cp /opt/ganglia/gmond/gmond.init /etc/rc.d/init.d/gmond;
    chkconfig --add gmond;
    chkconfig --list gmond;
Configure GMETAD:
  1. If this is a new installation then you can find the default gmetad.conf file in gmetad directory where you installed ganglia. In this case it would be 
    /opt/ganglia/gmetad/gmetad.conf
    . Copy this file to 
    /etc/ganglia/gmetad.conf
    . 
    cp /opt/ganglia/gmetad/gmetad.conf /etc/ganglia/gmetad.conf
  2. Create a symbolic link in 
    /usr/local/etc
    to the file you just copied: 
    ln -s /etc/ganglia/gmetad.conf /usr/local/etc/gmetad.conf;
  3. Modify gmetad.conf and include all the clusters you wish to include in this installation.
  4. Change the init file to point to correct binary: 
    vi /opt/ganglia/gmetad/gmetad.init
    change 
    GMETAD
    to 
    /usr/local/sbin/gmetad
    .
  5. Copy gmetad.init to the init.d directory and add to start up: 
    cp /opt/ganglia/gmetad/gmetad.init /etc/rc.d/init.d/gmetad;
    chkconfig --add gmetad;
    chkconfig --list gmetad;
Install Ganglia Web
  1. Decompress Ganglia Web gz file to /usr/share directory: 
    tar --xvzf ganglia--web-3.5.7.tar.gz <del>C /usr/share
  2. Change directory to /usr/share and rename gangliaweb-3.5.7 to ganglia
    cd /usr/share;<br />mv ganglia-web-3.5.7 ganglia;
  3. There are two directories required for dwoo module. Create them and change their ownership/permission as stated below: 
    mkdir /usr/share/ganglia/dwoo/compiled
    mkdir /usr/share/ganglia/dwoo/cache
    chown -R 1000:1000 *
    chmod -R 777 *
Configure Ganglia Web
  1. Set correct path for where ganglia conf directory is: 
    vi /usr/share/ganglia/conf_default.php
    Change $conf[‘gweb_confdir’] to the web folder. e.g. 
    $conf['gweb_confdir'] = "/usr/share/ganglia";
  2. Add Ganglia web folder to Apache HTTPD:
    • Create a new conf file in /etc/httpd/conf.d 
      vi /etc/httpd/conf.d/ganglia.conf
      The following is a sample you can use: 
      <location /ganglia>
Order deny,allow
Allow from all
</location>
Start all services
  1. Start gmetad service 
    service gmetad start
    Check /var/log/messages.log to make sure there aren’t any errors.
  2. Start GMETAD service 
    service gmond start
    Check /var/log/messages.log to make sure there aren’t any errors.
  3. Restart HTTPD service 
    service httpd restart

No Comments