The very first line of defence is an Intrusion Detection System. Host-based systems apply their detection at the host level and will typically detect most intrusion attempts quickly and notify you immediately so you can remedy the situation.
Open Source Security, or OSSEC, is by far the leading open-source host-based intrusion detection system. The product is owned by Trend Micro, one of the leading names in IT security and maker of one of the best virus protection suites. When installed on Unix-like operating systems, the software primarily focuses on log and configuration files. It creates checksums of important files and periodically validates them, alerting you whenever something odd happens. It will also monitor and alert on any abnormal attempt at getting root access. On Windows hosts, the system also keeps an eye for unauthorized registry modifications which could be a tell-tale sign of malicious activity.
By virtue of being a host-based intrusion detection system, OSSEC needs to be installed on each computer you want to protect. However, a centralized console does consolidate information from each protected computer for easier management. While the OSSEC console only runs on Unix-Like operating systems, an agent is available to protect Windows hosts. Any detection will trigger an alert which will be displayed on the centralized console while notifications will also be sent by email.
Samhain is another well-known free host intrusion detection system. Its main features, from an IDS standpoint, are file integrity checking and log file monitoring/analysis. It does way more than that, though. The product will perform rootkit detection, port monitoring, detection of rogue SUID executables, and of hidden processes. The tool was designed to monitor multiple hosts running various operating systems while providing centralized logging and maintenance. However, Samhain can also be used as a stand-alone application on a single computer. The software primarily runs on POSIX systems like Unix, Linux or OS X. It can also run on Windows under Cygwin, a package that allows running POSIX applications on Windows, although only the monitoring agent has been tested in that configuration.
One of Samhain’s most unique feature is its stealth mode which allows it to run without being detected by potential attackers. Intruders have been known to quickly kill detection processes they recognize as soon as they enter a system before being detected, allowing them to go unnoticed. Samhain uses steganographic techniques to hide its processes from others. It also protects its central log files and configuration backups with a PGP key to prevent tampering.
Sagan, which is actually more of a log analysis system than a true IDS. It has, however, some IDS-like features which is why it deserves a place on our list. The tool locally watches the log files of the system where it’s installed but it can also interact with other tools. It could, for instance, analyze Snort’s logs, effectively adding the NIDS functionality of Snort to what is essentially a HIDS. It won’t just interact with Snort. Sagan can interact with Suricata as well and it is compatible with several rule building tools like Oinkmaster or Pulled Pork.
Sagan also has script execution capabilities which can make it a crude intrusion prevention system, provided that you develop some remediation scripts. Although this tool might not likely be used as your sole defence against intrusion, it can be a great component of a system that can incorporate many tools by correlating events from different sources.