Infrastructure monitoring is a field that has so many solutions… from Zabbix to Nagios to dozens of other open-source tools. Despite the fact that there are now much newer kids on the block, Nagios is a veteran monitoring solution that is highly effective because of the large community of contributors who create plugins for the tool. Nagios does not include all the abilities that we had wanted around the automatic discovery of new instances and services, so we had to work around these issues with the community’s plugins. Fortunately, it wasn’t too hard, and Nagios works great.
We also looked into Icinga, which was originally created as a fork of Nagios. Its creators aim to take Nagois to the next level with new features and a modern user experience. There is a debate within the open source community about the merits of Nagios and its stepchild, but for now we are continuing to use Nagios and are satisfied with its scale and performance. The switch to newer technology, such as Icinga, may be appropriate in the future as we progress.
Sometimes the simplest tools are the most useful, as proven by the simple watchdog Monit. Its role is to ensure that any given process on a machine is up and running appropriately. For example, a failure occurs in Apache, Monit will help to restart the Apache process. It is very easy to setup and configure and is especially useful for multi-service architecture with hundreds of micro-services. If you are using Monit, make sure to monitor the restarts that it executes in order to surface problems and implement solutions (rather than just restarting and ignoring the failure). You can do this by monitoring Monit’s log files and ensuring that you are alerted to every restart.
The ELK Stack is the most common log analytics solution in the modern IT world. It collects logs from all services, applications, networks, tools, servers, and more in an environment into a single, centralized location for processing and analysis. We use it for analytical purposes (e.g., to troubleshoot problems, monitor services, and reduce the time it takes to solve operational issues). Another use for this tool is for security and auditing (e.g., to monitor changes in security groups and changes in permissions). After receiving alerts on these issues, it is easy to act on unauthorized users and activities. We also use ELK for business intelligence, such as monitoring our users and their behavior. You can set up your own ELK or buy it as-a-service. We’ve written a guide for the community on using ELK to monitor your application performance.
Disclaimer: Logz.io is our ELK-as-a-service that we use in our own environment. You can say that we eat our own dog food.
Consul is a great fit for service discovery and configuration in modern, elastic applications that are built from microservices. The open-source tool makes use of the latest technology in providing internal DNS names for services. It acts as a kind of broker to help you sign and register names, enabling you to access service names instead of specific machines. If, for example, you have a cluster of multiple machines, you can simply register them as a single entity under Consul and access the cluster easily. We praise this tool for its efficiency, although we still feel there is more that can be done with it. If you also use it, it would be great to hear about your own use case.
Everyone knows Jenkins, right? It’s not the fastest or the fanciest, but it’s really easy to start to use and it has a great ecosystem of plugins and add-ons. It is also optimized for easy customization. We have configured Jenkins to build code, create Docker containers (see the next item), run tons of tests, and push to staging/production. It’s a great tool, but there are some issues regarding scaling and performance (which isn’t so unusual). We’ve explored other cool solutions such as Travis and CircleCI, which are both hosted solutions that don’t require any maintenance on our side. For now, however, since we’ve already invested in Jenkins, we’ll continue with it.
Everything that can be said about how Docker is transforming IT environments has already been said. It’s great…life changing, even — (although we’re still experiencing some challenges with it). We use Docker in production for most services. It eases configuration management, control issues, and scaling by allowing containers to be moved from one place to another.
We have developed our SaaS solution with a twelve-layer pipeline of data processing. Together with Jenkins and Docker, we have been able to run a full pipeline across all layers on a single Mac. It would be wrong to say that there aren’t any complications with Docker, as even small containers can take a significant amount of time to build. However, we want to ensure that our developers are as satisfied as possible and enable them to work rapidly. With all of the management involved in storage, security, networking — and everything surrounding containers — this can be a challenge.
We see Docker progressing and look forward to welcoming the company’s new management and orchestration solutions. For those who might be having issues with Docker, we’ve also compiled a list ofchallenges and solutions when migrating to Docker.
Again, simplicity is key. Ansible is a configuration management tool that is similar to Puppet and Chef. Personally, we found those two to have more overhead and complexity to our use case– so we decided to go with Ansible instead. We know that Puppet and Chef probably have a richer feature set, but simplicity was our desired KPI here. We see some tradeoffs between configuration management using Ansible and the option to simply kill and spin new application instances using a Docker container. With Docker, we almost never upgrade machines but opt to spin new machines instead, which reduces the need to upgrade our EC2 cloud instances. Ansible is used mostly for deployment configuration mostly. We use it to push changes and re-configure newly-deployed machines. In addition, its ecosystem is great, with an easy option to write custom applications.
Collectd/l are nifty little tools that gather and store statistics about the system on which they run and are much more flexible than other tools. They allow users to measure the values of multiple system metrics and unlike other log collection tools that are designed to measure specific system parameters, Collectd/l can monitor different parameters in parallel. We use these two tools to measure customer performance parameters and ship them to our ELK-as-a-Service platform. We’ve specifically wrapped a Collectl agent in a Docker container and push it with Ansible to all of our servers. It collects information every couple of seconds and then ships it to ELK to allow us to run reports and send alerts. If you’d like to see a specific example of how we do this process in our environment and how others can do the same, we’ve created a guide for everyone.
Git was created 10 years ago following the Linux community’s need for SCM (Source Control Management) software that could support distributed systems. Git is probably the most common source management tool available today. After running Git internally for a short period of time, we realized that we were better suited with GitHub. In addition to its great forking and pull request features, GitHub also has plugins that can connect with Jenkins to facilitate integration and deployment. I assume that mentioning Git to modern IT teams is not breaking news, but I decided to add to it to the list due to its great value to us.
1. Kubernetes is an open source orchestration system for Docker containers. It handles scheduling onto nodes in a compute cluster and actively manages workloads to ensure that their state matches the users declared intentions. Using the concepts of “labels” and “pods”, it groups the containers which make up an application into logical units for easy management and discovery.Originally developed and owned by Google,Kubernetes was offered to the Cloud Native Computing Foundationto seed other complementary technologies to enable a set of common container technologies. It is an integral part of CoreOS’s Tectonic product, in addition to being tightly coupled with Mesos and the Mesosphere DCOS.Kubernetes offers the promise of Google-like container usage for everyone. It seems no matter what the basic container technology we use, it is being used with many of the container-based PaaS stacks today
2. Chef – From the official what is: “Chef is a systems and cloud infrastructure automation framework that makes it easy to deploy servers and applications to any physical, virtual, or cloud location, no matter the size of the infrastructure. Each organization is comprised of one (or more) workstations, a single server, and every node that will be configured and maintained by the chef-client. Cookbooks (and recipes) are used to tell the chef-client how each node in your organization should be configured. The chef-client (which is installed on every node) does the actual configuration.
More than the above though, for many Chef along with Puppet Labs and Jenkins are the heart and soul of DevOps tools. Moving beyond its original configuration management mission today Chef represents the move to “everything as code” while still maintaining its open source commitment. Chef Delivery, Chef Compliance move Chef into the realm of a tool that is involved in the entire DevOps workflow.
3. Puppet – The Romulus to Chef’s Remus, Puppet Labs is one of the founding members of the DevOps movement. Built on open source and still fully supporting its open source heritage going forward, Puppet like Chef has moved beyond its original mission of configuration management and automation of single machines or instances to orchestration of your entire stack.
From Puppet’s open source doc, Why Puppet
Puppet has been developed to help the sysadmin community move to building and sharing mature tools that avoid the duplication of everyone solving the same problem. It does so in two ways:
This means that your work as a sysadmin can get done much faster, because you can have Puppet handle most or all of the details, and you can download code from other sysadmins to help you get done even faster. The majority of Puppet implementations use at least one or two modules developed by someone else, and there are already hundreds of modules developed and shared by the community.
4. Maven – a Yiddish word meaning accumulator of knowledge, Maven was originally started as an attempt to simplify the build processes in the Jakarta Turbine project. There were several projects each with their own Ant build files that were all slightly different and JARs were checked into CVS. We wanted a standard way to build the projects, a clear definition of what the project consisted of, an easy way to publish project information and a way to share JARs across several projects.
The result is a tool that can now be used for building and managing any Java-based project. We hope that we have created something that will make the day-to-day work of Java developers easier and generally help with the comprehension of any Java-based project.
Maven’s primary goal is to allow a developer to comprehend the complete state of a development effort in the shortest period of time. In order to attain this goal there are several areas of concern that Maven attempts to deal with:
5. Gradle – the yin to Maven’s yang, Gradle and Maven are much like Puppet and Chef. In this case though Gradle was born out of frustration with Maven and other tools. From the Gradle, Inc. site:
Gradle was founded by Hans Dockter who was working in Enterprise software development and ran into an intractable build problem. At the time he was using Apache Maven. After three days and nights of exhaustive research, he discovered that the problem could not be fixed with the tool at all. Hans vowed to cure this problem for himself, for his team, his company, and for developers everywhere. He was joined in this mission by founder CTO Adam Murdoch. After releasing Gradle the open source project out into the wild in April of 2008, Hans and Adam then hired all of the key contributors whom they met along the way. These included world renown Gradle experts from companies like Thoughtworks, Typesafe, and Pivotal.
Many developers and shops today use one or the other of these open source tools.
6. Mattermost (& Zulip) Today is all about ChatOps. In many organizations it is replacing email as the primary communication method between teams and even outside members now. While there are plenty of commercial solutions in this space like HipChat and Slack, there are some worthy open source tools available as well. One of the biggest advantages to these open source solutions is you don’t have to use a hosted version if you are worried about control, governance or security issues. You can install these on your own server and maintain yourself if that is your preference. Here are two of the leading open source ChatOps tools:
Mattermost – is an open source Slack alternative. The company behind it is a Y Combinator funded startup with founders who come from Microsoft Office engineering. Frustrated that in using a SaaS hosted chat program they were locked out of their own data unless they paid to use the app they set out to create their own self-hosted app and so Mattermost was born. Not sure where the commercial element ill join this, perhaps open core type premium features? Support? Time will tell.
Zulip – Acquired and then released as open source under an Apache license by DropBox, Zulip is a modern take on ChatOps. Not based on IRC and XMPP, Zulip features: Threaded group conversations, One-on-one and group private conversations, Persistence, History, Full-history search, Team presence and buddy list, Inline image, video, and tweet previews, Drag-and-drop file uploads, Stream-wide announcements, Emails for important missed messages, Desktop notifications, Audible notifications, Hotkeys, Emoji, Code, Lightweight markup, Message editing, Invite-only streams, Starred messages, Integrations, API, Mobile apps, Desktop apps
7. OWASP – A developers and DevOps best security friend, OWASP has become the place for web application security. Their mission is to make software security visible, so that individuals and organizations worldwide can make informed decisions about true software security risks. OWASP is actually an umbrella for a number of different projects including tools, lists (OWASP top 10) and groups. Their website says it best:
The OWASP Foundation came online on December 1st 2001 it was established as a not-for-profit charitable organization in the United States on April 21, 2004 to ensure the ongoing availability and support for our work at OWASP. OWASP is an international organization and the OWASP Foundation supports OWASP efforts around the world. OWASP is an open community dedicated to enabling organizations to conceive, develop, acquire, operate, and maintain applications that can be trusted. All of the OWASP tools, documents, forums, and chapters are free and open to anyone interested in improving application security. We advocate approaching application security as a people, process, and technology problem because the most effective approaches to application security include improvements in all of these areas. We can be found at www.owasp.org.
OWASP is a new kind of organization. Our freedom from commercial pressures allows us to provide unbiased, practical, cost-effective information about application security. OWASP is not affiliated with any technology company, although we support the informed use of commercial security technology. Similar to many open-source software projects, OWASP produces many types of materials in a collaborative and open way. The OWASP Foundation is a not-for-profit entity that ensures the project’s long-term success.
8. Zabbix (& Observium) – OK so lots of people have grown to hate Nagios. There are still lot of people who use Nagios too. That being said, we wanted to highlight some open source options in the network and application monitoring space. Two we like are Zabbix and Observium.
Zabbix – The basic work sphere of Zabbix LLC is development of open source software for monitoring of networks and applications. Apart from that the company offers a wide range of professional services designed to fit every customer’s unique business demands including implementation, integration, custom development and consulting services as well as various training programs.
Zabbix team’s mission is to make a superior monitoring solution available and affordable for all.
The company’s flagship product is Zabbix, one of the most popular open source monitoring software in the world. It is already used by a vast number of companies, who have chosen it due to real scalability, high and robust performance, ease of use and extremely low costs of ownership.
Observium – is a low-maintenance auto-discovering network monitoring platform supporting a wide range of device types, platforms and operating systems including Cisco, Windows, Linux, HP, Juniper, Dell, FreeBSD, Brocade, Netscaler, NetApp and many more. Observium focuses on providing a beautiful and powerful yet simple and intuitive interface to the health and status of your network.
Professionally developed and maintained by a team of experienced network engineers and systems administrators, Observium is a platform designed and built by its users. Observium Community is available free to everyone and recieves updates and features twice annually. Observium Professionaladds priority access to daily updates and new features for a small yearly fee.
9. Cucumber – OK raise your hand if you use Cucumber. It is one of the best kept secrets in the business. I am constantly amazed out how many developers and DevOps folks mention this open source tool. It’s actually several tools as their are different versions for different languages with each one having a slightly different feature set.
Systems