Never Ending Security

It starts all here

OSSEC: Open-Source Host-Based Intrusion Detection System for Linux


OSSEC is an open-source, host-based intrusion detection system (HIDS) that performs log analysis, integrity checking, Windows registry monitoring, rootkit detection, time-based alerting, and active response. It can be used to monitor one server or thousands of servers in a server/agent mode. If properly configured, OSSEC can give you a real-time view into what’s happening on your server.

This tutorial will show you how to install and configure OSSEC to monitor Ubuntu server. We’ll configure OSSEC so that if a file is modified, deleted, or added to the server, OSSEC will notify you by email – in real-time. That’s in addition to other integrity-checking features that OSSEC offers.

OSSEC can do more than notify you of file modifications, but one article is not enough to show you how to take advantage of all its features.

You need to have a server that you want to monitor. This tutorial assumes that you already have one and that it’s already set up for use. The most important thing is that you have access to it and can log in via SSH. Setting up OSSEC is not something you want to undertake when you still don’t know how to ssh into your server.

  • Ubuntu server
  • You should create a sudo user on the server. In this example, the user is named root
  • Optional: If you want to send mail from a local SMTP server, you should install Postfix for simple email sending
  • Installation of OSSEC involves some compiling, so you need gcc and make installed. You can install both by installing a single package called build-essential
  • You also need to install a package called inotify-tools, which is required for real-time alerting to work

Open-Source Host-Based Intrusion Detection System: OSSEC Open-Source Host-Based Intrusion Detection System: OSSEC Open-Source Host-Based Intrusion Detection System: OSSECTo install all required packages, first update the server:

apt-get update

The install the packages:

apt-get install build-essential inotify-tools

Download and Verify OSSEC

Since this a security article, we’re going to do a little extra work to verify that we’re installing valid software. The idea is that you generate the MD5 and SHA1 checksums of the downloaded OSSEC tarball and compare them with those in the checksum file. If they match, then you can assume that the tarball has not been tampered with.

At the time of writing, the latest server edition of OSSEC is version 2.8.1. To download it, type:

wget -U ossec

To download the checksum file, type:

wget -U ossec

To verify that both files are in place, type:

ls -l ossec*

You should see the files:


Now, let’s examine the checksum file with the cat command, like so:

cat ossec-hids-2.8.1-checksum.txt


MD5(ossec-hids-2.8.1.tar.gz)= c2ffd25180f760e366ab16eeb82ae382
SHA1(ossec-hids-2.8.1.tar.gz)= 0ecf1df09558dc8bb4b6f65e1fb2ca7a7df9817c

In the above output, the important parts are those to the right of the = sign. Those are the MD5 and SHA1 checksums of the tarball.

Now we’ll make sure the checksums we generate for the tarball match the checksums we downloaded.

To generate the MD5sum of the tarball, type:

md5sum ossec-hids-2.8.1.tar.gz

Expected output:

c2ffd25180f760e366ab16eeb82ae382  ossec-hids-2.8.1.tar.gz

Compare the generated MD5 checksum with the one in the checksum file. They should match.

Do the same for the SHA1 checksum by typing:

sha1sum  ossec-hids-2.8.1.tar.gz

Expected output:

0ecf1df09558dc8bb4b6f65e1fb2ca7a7df9817c  ossec-hids-2.8.1.tar.gz

If both match, you’re good to go.

Install OSSEC

OSSEC can be installed in server, agent, local or hybrid mode. This installation is for monitoring the server that OSSEC is installed on. That means a local installation.

Before installation can start, you have to extract the file. You do that by typing:

tar -zxf ossec-hids-2.8.1.tar.gz

After that, you should have a directory named ossec-hids-2.8.1. To start installation, you have to change (cd) into that directory, which you do by typing:

cd ossec-hids-2.8.1

To see the contents of the directory that you’re now in, use the ls command by typing:

ls -lgG

You should see these files and directories:

total 100
drwxrwxr-x  4  4096 Apr  9 23:34 active-response
-rw-rw-r--  1   542 Apr  9 23:34 BUGS
-rw-rw-r--  1   289 Apr  9 23:34 CONFIG
drwxrwxr-x  6  4096 Apr  9 23:34 contrib
-rw-rw-r--  1  3196 Apr  9 23:34 CONTRIBUTORS
drwxrwxr-x  4  4096 Apr  9 23:34 doc
drwxrwxr-x  4  4096 Apr  9 23:34 etc
-rw-rw-r--  1  1848 Apr  9 23:34 INSTALL
-rwxrwxr-x  1 32019 Apr  9 23:34
-rw-rw-r--  1 24710 Apr  9 23:34 LICENSE
-rw-rw-r--  1  1664 Apr  9 23:34
drwxrwxr-x 30  4096 Apr  9 23:34 src

The only file of interest to us in that listing is That’s the OSSEC installation script. To initiate installation, type:


You will be prompted to answer some installation questions.

The first task that will be required of you is the selection of the language. As shown in the output below, the default is English. Throughout the installation process, if you’re required to make a selection, any entry in square brackets is the default. If the default is what you want, press the ENTER key to accept the default. Other than having to type your email address, we recommend that you accept all the defaults — unless you know what you’re doing.

So if your language is English, press ENTER. Otherwise, type the two letters for your language and press ENTER.

  (en/br/cn/de/el/es/fr/hu/it/jp/nl/pl/ru/sr/tr) [en]:

After selecting the language, you should see this:

OSSEC HIDS v2.8 Installation Script -

 You are about to start the installation process of the OSSEC HIDS.
 You must have a C compiler pre-installed in your system.
 If you have any questions or comments, please send an e-mail
 to (or

  - System: Linux arrakis 3.13.0-36-generic
  - User: root
  - Host: arrakis

  -- Press ENTER to continue or Ctrl-C to abort. --

After pressing ENTER, you should get:

1- What kind of installation do you want (server, agent, local, hybrid or help)? local

Type local and press ENTER. You should get:

  - Local installation chosen.

2- Setting up the installation environment.

  - Choose where to install the OSSEC HIDS [/var/ossec]:

Accept the default and press ENTER. After that, you’ll get:

    - Installation will be made at  /var/ossec .

3- Configuring the OSSEC HIDS.

  3.1- Do you want e-mail notification? (y/n) [y]:

Press ENTER.

  - What's your e-mail address?

Type the email address where you want to receive notifications from OSSEC.

  - We found your SMTP server as:
  - Do you want to use it? (y/n) [y]:

--- Using SMTP server:

Press ENTER unless you have specific SMTP server settings you want to use.

Now’s time to let OSSEC know what checks it should be running. In response to any prompt from the script, accept the default by pressing ENTER.

ENTER for the integrity check daemon.

  3.2- Do you want to run the integrity check daemon? (y/n) [y]:

- Running syscheck (integrity check daemon).

ENTER for rootkit detection.

  3.3- Do you want to run the rootkit detection engine? (y/n) [y]:

- Running rootcheck (rootkit detection).

ENTER for active response.

  3.4- Active response allows you to execute a specific command based on the events received.  

   Do you want to enable active response? (y/n) [y]:

   Active response enabled.

Accept the defaults for firewall-drop response. Your output may show some IPv6 options – that’s fine.

  Do you want to enable the firewall-drop response? (y/n) [y]:

- firewall-drop enabled (local) for levels >= 6

   - Default white list for the active response:

   - Do you want to add more IPs to the white list? (y/n)? [n]:

You may add your IP address here, but that’s not necessary.

OSSEC will now present a default list of files that it will monitor. Additional files can be added after installation, so press ENTER.

3.6- Setting the configuration to analyze the following logs:
    -- /var/log/auth.log
    -- /var/log/syslog
    -- /var/log/dpkg.log

 - If you want to monitor any other file, just change
   the ossec.conf and add a new localfile entry.
   Any questions about the configuration can be answered
   by visiting us online at .

   --- Press ENTER to continue ---

By this time, the installer has all the information it needs to install OSSEC. Kick back and let the installer do its thing. Installation takes about 5 minutes. If installation is successful, you are now ready to start and configure OSSEC.

Note: One reason installation might fail is if a compiler is not installed. In that case, you’ll get an error like this:

5- Installing the system
 - Running the Makefile
./ 85: ./ make: not found

 Error 0x5.
 Building error. Unable to finish the installation.

If you get that error, then you need to install build-essential.

If installation succeeds, you should see this type of output:

 - System is Debian (Ubuntu or derivative).
 - Init script modified to start OSSEC HIDS during boot.

 - Configuration finished properly.

 - To start OSSEC HIDS:
                /var/ossec/bin/ossec-control start

 - To stop OSSEC HIDS:
                /var/ossec/bin/ossec-control stop

 - The configuration can be viewed or modified at /var/ossec/etc/ossec.conf

    ---  Press ENTER to finish (maybe more information below). ---

OSSEC is now installed. The next step is to start it.


By default OSSEC is configured to start at boot, but the first time, you’ll have to start it manually.

If you want to check its current status, type:

/var/ossec/bin/ossec-control status

Expected output:

ossec-monitord not running...
ossec-logcollector not running...
ossec-syscheckd not running...
ossec-analysisd not running...
ossec-maild not running...
ossec-execd not running...

That tells you that none of OSSEC’s processes are running.

To start OSSEC, type:

/var/ossec/bin/ossec-control start

You should see it starting up:

Starting OSSEC HIDS v2.8 (by Trend Micro Inc.)...
Started ossec-maild...
Started ossec-execd...
Started ossec-analysisd...
Started ossec-logcollector...
Started ossec-syscheckd...
Started ossec-monitord...

If you check the status again, you should get confirmation that OSSEC is now running.

/var/ossec/bin/ossec-control status

This output shows that OSSEC is running:

ossec-monitord is running...
ossec-logcollector is running...
ossec-syscheckd is running...
ossec-analysisd is running...
ossec-maild is running...
ossec-execd is running...

Right after starting OSSEC, you should get an email that reads like this:

OSSEC HIDS Notification.
2015 Apr 10 11:15:38

Received From: ossec2->ossec-monitord
Rule: 502 fired (level 3) -> "Ossec server started."
Portion of the log(s):

ossec: Ossec started.

That’s another confirmation that OSSEC is working and will send you email alerts whenever something it’s configured to monitor happens. Even when it is restarted, OSSEC will send you an email.

If you didn’t get this email right away, don’t worry. You may still need to tweak your email settings to make sure your OSSEC server’s emails can get through to your mail provider. This is especially true for some 3rd-party email service providers like Google and Fastmail.

Configure OSSEC for Real-time Alerts on File Modifications

OSSEC’s default directory is a chroot-ed (sandbox) environment that only a user with root (admin) privileges can access. A standard user cannot cd into /var/ossec or even list the files in it. As the root (or admin) user, however, you can.

So, cd into the installation directory by typing:

cd /var/ossec

To list the files in your new working directory, type:

ls -lgG

You should see these files and directories:

total 40
dr-xr-x---  3 4096 Apr 10 10:16 active-response
dr-xr-x---  2 4096 Apr 10 10:16 agentless
dr-xr-x---  2 4096 Apr 10 10:16 bin
dr-xr-x---  3 4096 Apr 10 10:16 etc
drwxr-x---  5 4096 Apr 10 10:16 logs
dr-xr-x--- 11 4096 Apr 10 10:16 queue
dr-xr-x---  4 4096 Apr 10 10:16 rules
drwxr-x---  5 4096 Apr 10 10:16 stats
dr-xr-x---  2 4096 Apr 10 10:16 tmp
dr-xr-x---  3 4096 Apr 10 10:16 var
  • OSSEC’s main configuration file is in the /var/ossec/etc directory.
  • Predefined rules are in the /var/ossec/rules directory
  • Commands used to manage OSSEC are in /var/ossec/bin
  • Take note of the /var/ossec/logs directory. If OSSEC ever throws an error, the/var/ossec/logs/ossec.log file in that directory is the first place to look

Main configuration file, /var/ossec/etc/ossec.conf

To access the main configuration file, you have to change into /var/ossec/etc. To do that, type:

cd /var/ossec/etc

If you do an ls while in that directory, you’ll see these files and directories:

ls -lgG


total 120
-r--r----- 1 97786 Apr 10 10:23 decoder.xml
-r--r----- 1  2842 Apr 10 10:23 internal_options.conf
-r--r----- 1  3519 Apr 10 10:23 localtime
-r--r----- 1  7752 Apr 10 10:23 ossec.conf
-rw-r----- 1    87 Apr 10 10:23 ossec-init.conf
drwxrwx--- 2  4096 Apr 10 10:23 shared

The main configuration file is /var/ossec/etc/ossec.conf.

Before modifying the file, make a backup copy, just in case. To make that copy, use the cp command like so:

cp /var/ossec/etc/ossec.conf /var/ossec/etc/ossec.conf.00

The idea is if your changes don’t work or mess up the system, you can revert to the copy and be back to normal. It’s the simplest disaster recovery practice that you should always take advantage of.

Now, open ossec.conf by using the nano editor.

nano /var/ossec/etc/ossec.conf

The configuration file is a very long XML file with several sections.

Email settings

Note: Email is finicky in general, especially if you are sending to a stricter mail provider like sending to a Gmail address. Check your spam, and tweak your settings if necessary.

The first configuration options you’ll see are the email credentials you specified during installation. If you need to specify a different email address and/or SMTP server, this is the place to do it.


By default, OSSEC sends 12 emails per hour, so you’ll not be flooded with email alerts. You can increase or decrease that value by adding the <email_maxperhour>N</email_maxperhour> setting to that section so that it reads:


Please replace N with the number of emails you want to receive per hour, between 1 and 9999.

Some third-party email service providers (Google and Fastmail, for example) will silently drop alerts sent by OSSEC if the <email_from> address does not contain a valid domain par. To avoid that, make sure that that email address contains a valid domain part.

If you don’t want to use an external email provider’s SMTP server, you can specify your own SMTP server, if you have one configured. If your SMTP server is running on the same system as OSSEC, change the<smtp_server>setting to localhost. For example:


If you still aren’t receiving expected emails from OSSEC, check the logs at/var/ossec/logs/ossec.log for mail errors.

Example mail errors:

2015/04/10 11:28:35 os_sendmail(1767): WARN: End of DATA not accepted by server
2015/04/10 11:28:35 ossec-maild(1223): ERROR: Error Sending email to (smtp server)

You can use these error messages to help you debug any issues with receiving email notifications.

Frequency of scans

In the <syscheck> section of ossec.conf, which starts like this:

    <!-- Frequency that syscheck is executed - default to every 22 hours -->

We will turn on alerts for new file creation. Add the line <alert_new_files>yes</alert_new_files>so that it reads like this:

    <!-- Frequency that syscheck is executed - default to every 22 hours -->


For testing purposes, you may also want to set the frequency of the system check to be much lower. By default, the system check is run every 22 hours. For testing purposes, you may want to set this to once a minute, that is,60 seconds. Revert this to a sane value when you are done testing.

    <!-- Frequency that syscheck is executed - default to every 22 hours -->


Directory and file change settings

Right after that, you should see the list of system directories that OSSEC monitors. It reads like:

<!-- Directories to check  (perform all possible verifications) -->
<directories check_all="yes">/etc,/usr/bin,/usr/sbin</directories>
<directories check_all="yes">/bin,/sbin</directories>

Let’s enable real-time monitoring by adding the settings report_changes="yes" realtime="yes" to each line. Modify these lines so they read:

<!-- Directories to check  (perform all possible verifications) -->
<directories report_changes="yes" realtime="yes" check_all="yes">/etc,/usr/bin,/usr/sbin</directories>
<directories report_changes="yes" realtime="yes" check_all="yes">/bin,/sbin</directories>

report_changes="yes" does exactly what is says. Ditto for realtime="yes".

In addition to the default list of directories that OSSEC has been configured to monitor, you can add new directories that you wish to monitor. In this next section, we are telling OSSEC to monitor /root and/var/www. We are going to add a new line right under the existing ones, so that that section now reads:

<!-- Directories to check  (perform all possible verifications) -->
<directories report_changes="yes" realtime="yes" check_all="yes">/etc,/usr/bin,/usr/sbin</directories>
<directories report_changes="yes" realtime="yes" check_all="yes">/bin,/sbin</directories>

<directories report_changes="yes" realtime="yes" restrict=".php|.js|.py|.sh|.html" check_all="yes">/root,/var/www</directories>

You should modify the directories to match your desired settings. If your user is not named root, you will want to change the path to the home directory.

For the new directories to monitor, we’ve added the restrict option, which tells OSSEC to monitor only the specified file formats. You don’t have to use that option, but it comes in handy when you have other files, like image files, that you don’t want OSSEC to alert on.

That’s all the changes for ossec.conf. You may save and close the file.

Local rules in /var/ossec/rules/local_rules.xml

The next file to modify is in the /var/ossec/rules directory, so cd into it by typing:

cd /var/ossec/rules

If you do an ls in that directory, you’ll see a bunch of XML files like these:

ls -lgG

Abbreviated output:

total 376
-r-xr-x--- 1  5882 Apr 10 12:13 apache_rules.xml
-r-xr-x--- 1  2567 Apr 10 12:13 arpwatch_rules.xml
-r-xr-x--- 1  3726 Apr 10 12:13 asterisk_rules.xml
-r-xr-x--- 1  4315 Apr 10 12:13 attack_rules.xml


-r-xr-x--- 1  1772 Apr 10 12:13 local_rules.xml


-r-xr-x--- 1 10359 Apr 10 12:13 ossec_rules.xml


Only two of those files are of interest to us now – local_rules.xml and ossec_rules.xml. The latter contains OSSEC’s default rule definitions, while the former is where you add your custom rules.

The default rule definitions in ossec_rules.xml are useful to look at so we can modify and copy them into our local rules. In ossec_rules.xml, the rule that fires when a file is added to a monitored directory is rule554. By default, OSSEC does not send out alerts when that rule is triggered, so the task here is to change that behavior. Here’s what rule 554 looks like in the default version:

<rule id="554" level="0">
<description>File added to the system.</description>

OSSEC does not send out an alert if a rule has a level set to 0. We want to modify this rule to raise the alert level. Instead of changing it in the default file, we will copy the rule to local_rules.xml and modify it so that it can trigger an alert.

To do that, make a backup copy of the /var/ossec/rules/local_rules.xml file:

cp /var/ossec/rules/local_rules.xml /var/ossec/rules/local_rules.xml.00

Edit the file with nano:

nano /var/ossec/rules/local_rules.xml

Add the new rule at the end of the file. Make sure that it is within the <group> ... </group> tag.

<rule id="554" level="7" overwrite="yes">
<description>File added to the system.</description>

Save and close the file.

Those are all the changes necessary.

Restart OSSEC

All that’s left now is to restart OSSEC, something that has to be done every time you modify OSSEC’s files. To restart OSSEC type:

/var/ossec/bin/ossec-control restart

If all is working correctly, you should receive an email from OSSEC informing you that it has (re)started.

Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s