Breaking News
Home / Tools / Dev & Build

Dev & Build

Development and build tooling

Apache Chukwa

AT A GLANCE Apache Chukwa is an open source data collection system for monitoring large distributed systems. Chukwa is built on top of the Hadoop Distributed File System (HDFS) and Map/Reduce framework and inherits Hadoop’s scalability and robustness. Chukwa also includes a toolkit for displaying, monitoring and analyzing results to …

Read More »

Apache Giraph

apache giraph

AT A GLANCE Apache’s Giraph project is said to be “a scalable, fault-tolerant implementation of graph-processing algorithms in Apache Hadoop clusters of up to thousands of computing nodes.” Giraph is in use at companies like Facebook and PayPal to help represent and analyze the billions (or even trillions) of connections …

Read More »

Apache Bigtop

BigTop

AT A GLANCE Apache’s Bigtop was originally developed and released as an open source packaging infrastructure by Cloudera. BigTop is used for some vendors to build their own distributions based on Apache Hadoop (CDH, Pivotal HD, Intel’s distribution), however Apache Bigtop does many more tasks, like continuous integration testing (with Jenkins, …

Read More »

LinkedIn Norbert

Norbert is a library that provides easy cluster management and workload distribution. With Norbert, you can quickly distribute a simple client/server architecture to create a highly scalable architecture capable of handling heavy traffic. Implemented in Scala, Norbert wraps ZooKeeper, Netty and uses Protocol Buffers for transport to make it easy …

Read More »

Apache Twill

Apache Twill

AT A GLANCE Apache’s Twill project is an abstraction over Apache Hadoop® YARN that reduces the complexity of developing distributed applications, allowing developers to focus more on their business logic. Twill uses a simple thread-based model that Java programmers will find familiar. YARN can be viewed as a compute fabric …

Read More »

Apache Whirr

Apache Whirr

AT A GLANCE The Apache Whirr project provides a Java API and set of shell scripts for installing and running various services on cloud providers such as Amazon EC2 and Rackspace. Whirr allows you to define the layout of a cluster in terms of the number of nodes as well …

Read More »