tag:blogger.com,1999:blog-85097444893478683362024-03-14T00:50:46.246-07:00Level up IRLLeveling up the skills to pay the bills in the RPG called Life.Rodolf Sabalburohttp://www.blogger.com/profile/06648192806453188759noreply@blogger.comBlogger14125tag:blogger.com,1999:blog-8509744489347868336.post-35965275970810090822015-02-15T11:47:00.000-08:002015-02-15T11:54:19.295-08:00ntopng 1.2.2 on Ubuntu 14.04, revisited<p>I was doing some random research on <code>ntopng</code> a few days ago and I happened to stumble upon a page that was linking to my own blog. I didn't ever think I'd see the day where someone would <a href="http://www.reddit.com/r/networking/comments/2mb9a4/installing_ntopng_on_ubuntu_running_into_a_wall/">link to one of my tutorials</a>, especially on a sub-Reddit. This blog was mainly created for myself, little things like this motivate me to post more often.</p>
<p>My <a href="http://rsabalburo.blogspot.ca/2014/07/ntopng-on-ubuntu-1404-lts-server.html">ntopng on Ubuntu 14.04 LTS Server</a> post was created only 7 months ago. Unfortunately my tutorial didn't work out for that individual user, and it goes to show how quickly documentation can become inconsistent, especially in the open source world. So I've decided to revisit the topic and redocument it again from scratch; below are directions for installing <code>ntopng</code> <strong>1.2.2</strong> on Ubuntu 14.04.</p>
<h3>Installing ntopng</h3>
<p>Directions for installing <code>ntopng</code> seem liked they are far more streamlined compared to when I first did this last July. I'll be simply following the ntop.org official directions for their <a href="http://www.nmon.net/apt-stable/">stable build packages</a>.</p>
<p>Pull down the <code>apt-ntop-stable.deb</code> package using <code>wget</code>, and install with <code>dpkg</code>:</p>
<pre>
ubuntu@ubuntu-14-04:~$ sudo -i
[sudo] password for ubuntu:
root@ubuntu-14-04:~# wget http://www.nmon.net/apt-stable/14.04/all/apt-ntop-stable.deb
root@ubuntu-14-04:~# ls
apt-ntop-stable.deb
root@ubuntu-14-04:~# dpkg -i apt-ntop-stable.deb
Selecting previously unselected package apt-ntop-stable.
(Reading database ... 55712 files and directories currently installed.)
Preparing to unpack apt-ntop-stable.deb ...
Unpacking apt-ntop-stable (2.1-288) ...
Setting up apt-ntop-stable (2.1-288) ...
Adding ntop key to apt keyring
OK
</pre>
<p>The <code>apt-ntop-stable.deb</code> package doesn't install <code>ntopng</code> itself, it's simply files to add the repository. See below:</p>
<pre>
root@ubuntu-14-04:~# dpkg -l | grep ntop
ii apt-ntop-stable 2.1-288 all ntop apt package repository
root@ubuntu-14-04:~# dpkg -L apt-ntop-stable
/.
/etc
/etc/nbox
/etc/nbox/ntop-apt.key
/etc/apt
/etc/apt/sources.list.d
/etc/apt/sources.list.d/ntop-stable.list
</pre>
<p>Run <code>apt-get update</code> to update your system repositories and install the packages as <a href="http://www.nmon.net/apt-stable/">per directions</a>:</p>
<pre>
root@ubuntu-14-04:~# apt-get update
root@ubuntu-14-04:~# apt-get -y install pfring nprobe ntopng ntopng-data n2disk nbox
</pre>
<p>Here's a short description of what each package in the family does:</p>
<pre>
root@ubuntu-14-04:~# dpkg -s pfring nprobe ntopng ntopng-data n2disk nbox | egrep "^Package|^Description"
Package: pfring
Description: PF_RING (http://www.ntop.org/pf_ring/)
Package: nprobe
Description: A network probe.
Package: ntopng
Description: Web-based traffic monitoring.
Package: ntopng-data
Description: Data files (geoip) for ntopng.
Package: n2disk
Description: A packet-to-disk application.
Package: nbox
Description: Web management interface for ntop apps.
</pre>
<ul>
<li><a href="http://www.ntop.org/products/pf_ring/">pfring</a> is a module that allows for high-speed package captures, it's recommended to enable this if you plan on capturing on high-traffic interfaces.</li>
<li><a href="http://www.ntop.org/products/nprobe/">nProbe</a> is simply the NetFlow probe, for example you can setup multiple probes throughout your network and send all the NetFlow data to a central <code>ntopng</code> instance to visualize all the traffic.</li>
<li><a href="http://www.ntop.org/products/n2disk/">n2disk</a> allows you to efficiently write huge volumes of packet captures to disk without packet loss.</li>
</ul>
<p>A lot of packages will be installed, and at the very end you should see the following message:</p>
<pre>
IMPORTANT IMPORTANT IMPORTANT IMPORTANT IMPORTANT
You can now point your browser to https://localhost/
The default user is nbox with password nbox
IMPORTANT IMPORTANT IMPORTANT IMPORTANT IMPORTANT
</pre>
<ul>
<li>Don't forget that the address is https and <em>not</em> regular https. If you use http it will direct you to the Apache2 Ubuntu Default Page.</li>
</ul>
<p>Before you visit the https://localhost page, however, restart the <code>apache2</code> service:</p>
<pre>
root@ubuntu-14-04:~# service apache2 restart
* Restarting web server apache2 AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName' directive globally to suppress this message
</pre>
<p>I was getting a <em>Service Unavailable</em> error, and the restart fixed this.</p>
<h3>Starting ntopng with the nBox web GUI</h3>
<p>Unlike before where we had to configure things manually, the new ntop UI or nBox web GUI makes many of the configurations trivial.</p>
<p>After logging into https://localhost with the default credentials (<code>nbox:nbox</code>), at the top of the nBox dashboard:</p>
<ul>
<li>Applications > ntopng</li>
<li>Under the Configuration > General</li>
<li>Select which interfaces you want to monitor. If you want to select multiple, hold <em><Ctrl></em> and click.</li>
<li>Enable the service to startup automatically, if needed.</li>
</ul>
<p>You can edit other settings under Hosts, Flows, Directory, and Advanced.</p>
<p>When finished, click Save Changes. Then click back to the Status tab, and click On for the interface you selected.</p>
<p>The interface will tell you that you can now access <code>ntopng</code> at the <strong>http://<server IP>:3000</strong> address.</p>
<p>Note that additional changes under the Configuration tab first require you to stop the <code>ntopng</code> service by clicking the Off button for your interface under the Status tab.</p>
<h3><p>Change those default credentials</p></h3>
<h4>nBox GUI</h4>
<p>On the of the nBox web GUI, System > Users. Web Users > for the already selected nbox user click Change Pwd.</p>
<h4>ntopng GUI</h4>
<p>The default username and password for the <code>ntopng</code> web interface is <code>admin</code>. To change the defaults, after logging into the web interface, click the Gear Icon > Manage Users, for the <code>admin</code> user, click Manage and change the password.</p>
<h3>Conclusion</h3>
<p>The nBox web UI greatly simplified configuration of all the components of the ntop family, in addition, it's refreshing to see that the ntopng UI is becoming more and more refined with each release.</p>
Rodolf Sabalburohttp://www.blogger.com/profile/06648192806453188759noreply@blogger.com7tag:blogger.com,1999:blog-8509744489347868336.post-47677256506933702042015-02-09T13:07:00.001-08:002015-02-09T13:09:16.663-08:00PPPoE on Ubuntu 14.04<p>It took me over the course of a week and probably close to 20 hours of troubleshooting to finally get my Linux router to establish a PPPoE connection to my provider. There were two major hurdles I overcame:</p>
<ol>
<li>The first was figuring out that the VLAN going to the DSL modem needed to be manually tagged. I found out this out by connecting my Windows laptop and tagging it's VLAN on the network adapter after seeing that it was also tagged in the router they used to setup my initial connection. I suspect they do this to differentiate between the IP phone and cable box traffic.</li>
<li>The second hurdle was <em>properly</em> configuring PPPoE properly in Linux. I emphasize properly because PPPoE in Linux is one of those topics that are barely documented, or if it is documented its done 50 different ways, all which don't work correctly for you — kind of like LDAP.</li>
</ol>
<h3>Adding VLAN support</h3>
<p>Note this part may not be needed, double check your settings on a working router to see if the VLAN is configured for the WAN interface. If you are able to receive a DHCP lease, but can't establish a PPPoE connection or see any response in the PPPoE logs, you may need to tag the VLAN on your network interface.</p>
<p>This is the part which probably took up the majority of my time because when I ran the <code>pppoeconf</code> utility (like many tutorials and StackOverflow responses tell you to do), it would simply hang. I would suggest trying to use <code>pppoeconf</code> first to configure PPPoE (there are several tutorials out there), if that doesn't work for you, try the procedures outlined in this tutorial.</p>
<p>Tagging VLANs isn't supported by default in Ubuntu 14.04, luckily, the <code>vlan</code> package can do that for us. Install the the <code>vlan</code> package:</p>
<pre>
root@ubuntu:~# aptitude -yvV install vlan
</pre>
<p>Load the <code>8021q</code> module and verify it is loaded:</p>
<pre>
root@ubuntu:~# modprobe 8021q
root@ubuntu:~# lsmod | grep ^8021q
8021q 24712 0
</pre>
<p>Ensure this module is loaded each time at boot by appending it to <code>/etc/modules</code>:</p>
<pre>
root@ubuntu:~# echo "8021q" >> /etc/modules
</pre>
<p>Use the <code>vconfig</code> utility to add a VLAN to the specified interface:</p>
<pre>
root@ubuntu:/etc/network# vconfig add eth0 20
Added VLAN with VID == 20 to IF -:eth0:-
</pre>
<ul>
<li>In the above I tagged my <code>eth0</code> interface with VLAN 20.
</ul>
<p>As additional verification you can <code>cat</code> the contents of <code>/proc/net/vlan/config</code> which would give you output similar to the following:</p>
<pre>
root@ubuntu:/etc/network# cat /proc/net/vlan/config
VLAN Dev name | VLAN ID
Name-Type: VLAN_NAME_TYPE_RAW_PLUS_VID_NO_PAD
eth0.20 | 20 | eth0
</pre>
<p>This will allow you to reference the VLAN-tagged interface as <code>eth0.20</code>. As another example, if you were to tag your <code>eth1</code> interface with VLAN 50 the interface name would be <code>eth1.50</code>.</p>
<h3><p>Network interfaces</p></h3>
<p>After issuing <code>vconfig</code>, you should be able to create and reference the interface in the <code>/etc/network/interfaces</code> file.</p>
<pre>
#
# WAN interface
#
auto eth0
iface eth0 inet dhcp
#
# WAN interface tagged with VLAN 20
#
auto eth0.20
iface eth0.20 inet manual
vlan-raw-device eth0
</pre>
<p>Now if you have a working physical connection from your ISP facing interface to the DSL modem, you should be able to get a DHCP lease with an private IP address on the original interface, in this case <code>eth0</code>.</p>
<p>The trick was understanding that I had to tell my PPPoE client to use the <code>eth0.20</code> interface, not <code>eth0</code>, to send the initial request for the PPPoE connection. This is where the <code>pppoeconf</code> setup would hang for me.</p>
<h3>RP-PPPoE</h3>
<p>Unfortunately it's very easy to get confused between all the PPP/PPPoE packages, e.g. <code>ppp</code>, <code>pppconfig</code>, <code>pppoe</code>, <code>pppoeconf</code>, <code>rp-pppoe</code>, etc. Even worse, is they all dump their configuration files and scripts in the same place, the <code>/etc/ppp</code> directory.</p>
<p>The client that I ended up using was <code>rp-pppoe</code> by <a href="https://www.roaringpenguin.com/products/pppoe">Roaring Penguin Software</a>.</p>
<p>Download the tar archive from their website:</p>
<pre>
root@ubuntu:~# wget https://www.roaringpenguin.com/files/download/rp-pppoe-3.11.tar.gz
2015-02-07 17:15:55 (167 KB/s) - ‘rp-pppoe-3.11.tar.gz’ saved [223234/223234]
</pre>
<p>Install the <code>build-essential</code> package which contains additional utilities needed to build packages from source:</p>
<pre>
root@ubuntu:~# aptitude -yvV install build-essential
</pre>
<p>Unarchive, change into the unarchived directory, and run the <code>./go</code> script:</p>
<pre>
root@ubuntu:~# tar xvf rp-pppoe-3.11.tar.gz
root@ubuntu:~# cd rp-pppoe-3.11/
root@ubuntu:~/rp-pppoe-3.11# ls
configs doc go go-gui gui man README rp-pppoe.spec scripts SERVPOET src
root@ubuntu:~/rp-pppoe-3.11# ./go
</pre>
<p>Now if <code>rp-pppoe</code> compiled correctly it should kick off a script immediately afterwards that prompts you for input:</p>
<pre>
Welcome to the Roaring Penguin PPPoE client setup. First, I will run
some checks on your system to make sure the PPPoE client is installed
properly...
Looks good! Now, please enter some information:
USER NAME
>>> Enter your PPPoE user name (default bxxxnxnx@sympatico.ca): dsluser
INTERFACE
>>> Enter the Ethernet interface connected to the DSL modem
For Solaris, this is likely to be something like /dev/hme0.
For Linux, it will be ethn, where 'n' is a number.
(default eth0): eth0.20
Do you want the link to come up on demand, or stay up continuously?
If you want it to come up on demand, enter the idle time in seconds
after which the link should be dropped. If you want the link to
stay up permanently, enter 'no' (two letters, lower-case.)
NOTE: Demand-activated links do not interact well with dynamic IP
addresses. You may have some problems with demand-activated links.
>>> Enter the demand value (default no):
DNS
Please enter the IP address of your ISP's primary DNS server.
If your ISP claims that 'the server will provide DNS addresses',
enter 'server' (all lower-case) here.
If you just press enter, I will assume you know what you are
doing and not modify your DNS setup.
>>> Enter the DNS information here: server
PASSWORD
>>> Please enter your PPPoE password:
>>> Please re-enter your PPPoE password:
FIREWALLING
Please choose the firewall rules to use. Note that these rules are
very basic. You are strongly encouraged to use a more sophisticated
firewall setup; however, these will provide basic security. If you
are running any servers on your machine, you must choose 'NONE' and
set up firewalling yourself. Otherwise, the firewall rules will deny
access to all standard servers like Web, e-mail, ftp, etc. If you
are using SSH, the rules will block outgoing SSH connections which
allocate a privileged source port.
The firewall choices are:
0 - NONE: This script will not set any firewall rules. You are responsible
for ensuring the security of your machine. You are STRONGLY
recommended to use some kind of firewall rules.
1 - STANDALONE: Appropriate for a basic stand-alone web-surfing workstation
2 - MASQUERADE: Appropriate for a machine acting as an Internet gateway
for a LAN
>>> Choose a type of firewall (0-2): 0
** Summary of what you entered **
Ethernet Interface: eth0.20
User name: dsluser
Activate-on-demand: No
DNS addresses: Supplied by ISP's server
Firewalling: NONE
>>> Accept these settings and adjust configuration files (y/n)? y
Adjusting /etc/ppp/pppoe.conf
Adjusting /etc/ppp/pap-secrets and /etc/ppp/chap-secrets
(But first backing it up to /etc/ppp/pap-secrets-bak)
(But first backing it up to /etc/ppp/chap-secrets-bak)
Congratulations, it should be all set up!
Type 'pppoe-start' to bring up your PPPoE link and 'pppoe-stop' to bring
it down. Type 'pppoe-status' to see the link status.
</pre>
<ul>
<li>This script will create the <code>/etc/ppp/pppoe.conf</code> populated with configuration parameters that you provided as input.</li>
<li>It will also put the username and password into the <code>/etc/ppp/pap-secrets</code> and <code>/etc/ppp/chap-secrets</code> file for you.</li>
</ul>
<p>Again, note that I specified my VLAN-tagged interface <code>eth0.20</code> as my interface connected to the DSL modem, and not <code>eth0</code>.</p>
<p>Now you should be able to run <code>pppoe-start</code>:</p>
<pre>
root@ubuntu:~/rp-pppoe-3.11# pppoe-start
. Connected!
</pre>
<p>The <code>plog</code> command will show you logging information from the initiated connection:</p>
<pre>
root@d54250wyk:~# plog
Feb 7 15:32:08 ubuntu pppd[2134]: Remote message: Login ok
Feb 7 15:32:08 ubuntu pppd[2134]: PAP authentication succeeded
Feb 7 15:32:08 ubuntu pppd[2134]: not replacing existing default route via 10.150.32.1
Feb 7 15:32:08 ubuntu pppd[2134]: local IP address 176.205.250.149
Feb 7 15:32:08 ubuntu pppd[2134]: remote IP address 31.215.80.1
Feb 7 15:32:08 ubuntu pppd[2134]: primary DNS address 213.42.20.20
Feb 7 15:32:08 ubuntu pppd[2134]: secondary DNS address 195.229.241.222
</pre>
<p>You should now have a <code>ppp0</code> interface, use <code>ifconfig</code> or <code>ip addr ls</code> to verify it is there.</p>
<p>In the above output you can see the message <code>not replacing existing default route via 10.150.32.1</code>. This was the default route that was obtained from the original DHCP lease on the <code>eth0</code> interface. There was a bug in the <code>rp-pppoe</code> utility that regardless of settings in the <code>/etc/ppp/pppoe.conf</code> file, it would not obtain and correctly replace the default route via the <code>ppp0</code> interface.</p>
<p>To resolve this issue, and correctly obtain the default route upon connect, delete all the files in the <code>/etc/ppp/peers/</code> directory:</p>
<pre>
root@ubuntu:~# rm -v /etc/ppp/peers/*
removed ‘/etc/ppp/peers/dsl-provider’
removed ‘/etc/ppp/peers/dsl-provider.dpkg-old’
removed ‘/etc/ppp/peers/provider’
</pre>
<p>Run <code>pppoe-stop</code> and <code>pppoe-start</code> to connect again:</p>
<pre>
root@ubuntu:/etc/openvpn# pppoe-stop
Killing pppd (2769)
Killing pppoe-connect (2749)
root@ubuntu:/etc/openvpn# pppoe-start
. Connected!
</pre>
<p>After your connection is established and verified, your configurations for all your network applications, such as <code>iptables</code> should reference the <code>ppp0</code> interface as your primary interface.</p>
Rodolf Sabalburohttp://www.blogger.com/profile/06648192806453188759noreply@blogger.com0tag:blogger.com,1999:blog-8509744489347868336.post-31156311534138401862015-02-06T11:53:00.000-08:002015-02-06T12:17:42.806-08:00Configuring GeoIP support for Shorewall on Ubuntu 14.04<p>If you've run <code>fail2ban</code> on any of your servers, see <a href="http://rsabalburo.blogspot.com/2014/07/fail2ban-with-shorewall.html">fail2ban with Shorewall</a>, you'll quickly find out that a majority of the banned IPs will originate from many of the same countries, usually China. One of the techniques I previously used to block out an entire country's network range was to use <a href="https://en.wikipedia.org/wiki/Netfilter#ipset">ipsets</a>. I used custom bash scripts to pull down zone information from <a href="http://www.ipdeny.com/ipblocks/">http://www.ipdeny.com/ipblocks/</a> and import them into separate ipsets for countries I wanted. There were certain limitations with this solution, however, such as maintaining up to date ipsets across multiple servers, some issues with Shorewall losing the ipsets across reboots or restarts, and the US netblock space being to large to fit into a single ipset.</p>
<p>Shorewall version 4.5.4 <a href="http://shorewall.net/ISO-3661.html">introduced the ability to support GeoIP 2-character ISO 3166 country codes</a>. This method is far more efficient and easier to maintain as the GeoIP database holds all the netblocks for all the countries in the world in an offline hashed database.</p>
<h3>xt_geoip module installation</h3>
<p>The first step is to install the <code>xt_geoip</code> kernel module which allows you to reference two letter country codes (e.g. US, CN, UK, CA, FR, etc.) in <code>iptables</code> or Shorewall.</p>
<p>This module can be found in the <code>xtables-addons-common 2.3-1</code> package, but <strong>do not install this default package</strong> to use the <code>xt_geoip</code> module. I probably spent over 4 hours trying to figure out why my virtual machine was crashing every time a rule triggered the use of the <code>xt_geoip</code> module only to find out that there's a <a href="https://bugs.launchpad.net/ubuntu/+source/xtables-addons/+bug/1286911">verified bug that causes a kernel panic</a>.</p>
<h4>Patrick Domack PPA to the rescue</h4>
<p>Fortunately, if you read the bug comments, a user by the name of <a href="https://launchpad.net/~patrickdk">Patrick Domack</a> provided his own <a href="https://launchpad.net/~patrickdk/+archive/ubuntu/general-lucid">PPA</a> with a packaged version of <code>xtables-addons-common</code> version <code>2.6-1~ppa1</code> which fixes this bug.</p>
<p>Add the PPA with <code>apt-add-repository ppa:patrickdk/general-lucid</code>:</p>
<pre>
root@ubuntu:~# add-apt-repository ppa:patrickdk/general-lucid
Packages used for my personal productions systems where newer versions or special patches are needed.
More info: https://launchpad.net/~patrickdk/+archive/ubuntu/general-lucid
Press [ENTER] to continue or ctrl-c to cancel adding it
gpg: keyring `/tmp/tmp24yt3e8g/secring.gpg' created
gpg: keyring `/tmp/tmp24yt3e8g/pubring.gpg' created
gpg: requesting key 4D79B5B5 from hkp server keyserver.ubuntu.com
gpg: /tmp/tmp24yt3e8g/trustdb.gpg: trustdb created
gpg: key 4D79B5B5: public key "Launchpad PPA for Patrick Domack" imported
gpg: Total number processed: 1
gpg: imported: 1 (RSA: 1)
OK
</pre>
<p><code>apt-get update</code> to update the repository and verify the new version of <code>xtables-addons-common</code> is available:</p>
<pre>
root@ubuntu:~# apt-get update
root@ubuntu:~# apt-cache show xtables-addons-common
Package: xtables-addons-common
Source: xtables-addons
Priority: extra
Section: admin
Installed-Size: 326
Maintainer: Pierre Chifflier <pollux@debian.org>
Architecture: amd64
Version: 2.6-1~ppa1
</pre>
<p>Install <code>xtables-addons-common</code> and verify the correct version is installed:</p>
<pre>
root@ubuntu:~# aptitude -yvV install xtables-addons-common
root@ubuntu:~# dpkg -l | grep xtables-addons-common
ii xtables-addons-common 2.6-1~ppa1 amd64 Extensions targets and matches for iptables [tools, libs]
</pre>
<h3><p>Building the GeoIP database for xt_geoip</p></h3>
<p>The package will install two scripts in the <code>/usr/lib/xtables-addons</code> directory:</p>
<pre>
root@ubuntu:~# ls /usr/lib/xtables-addons/
xt_geoip_build xt_geoip_dl
</pre>
<ul><li>According to http://xtables-addons.sourceforge.net/geoip.php the <em>"<code>xt_geoip_dl</code> simply calls wget on the hardcoded URLs and unpacks the retrieved files into the current directory. Then use <code>xt_geoip_build</code> to transform the CSV into the packed format"</em></li></ul>
<p>The <code>xt_geoip_dl</code> script uses the <code>unzip</code> utility to unpack the a <code>.csv</code> file, install <code>unzip</code> if it isn't already:</p>
<pre>
root@ubuntu:~# aptitude -yvV install unzip
</pre>
<p>Change into <code>/tmp</code> and run <code>/usr/lib/xtables-addons/xt_geoip_dl</code>:</p>
<pre>
root@ubuntu:/tmp# /usr/lib/xtables-addons/xt_geoip_dl
</pre>
<p>Note that both <code>iptables</code> and <code>shorewall</code> will look for the GeoIP database in the <code>/usr/share/xt_geoip</code> directory, by default it is not created. Also, the build script requires the <code>libtext-csv-xs-perl</code> module to parse the <code>.csv</code> file.</p>
<p>Create the <code>/usr/share/xt_geoip</code> directory and install the required perl module:</p>
<pre>
root@ubuntu:/tmp# mkdir /usr/share/xt_geoip
root@ubuntu:/tmp# aptitude -yvV install libtext-csv-xs-perl
</pre>
<p>Run the build script to create the GeoIP database from the <code>.csv</code> files and place the them in the <code>/usr/share/xt_geoip</code> directory:</p>
<pre>
root@ubuntu:/tmp# /usr/lib/xtables-addons/xt_geoip_build -D /usr/share/xt_geoip *.csv
</pre>
<h3><p>Loading the module and configuring Shorewall</p></h3>
<p>Load the kernel module and verify it was loaded:</p>
<pre>
root@ubuntu:~# modprobe xt_geoip
root@ubuntu:~# lsmod | grep ^xt_geoip
xt_geoip 12775 0
</pre>
<p>For verification I'm going to add a rule to reject and log any ping from my virtual machine to any address in the United States. Note, I wouldn't recommend blocking all of U.S. in production, because the IP addresses to Google for other countries are still considered U.S. based. Though whitelisting the U.S. address space may be useful in some environments.</p>
<p>Edit the <code>/etc/shorewall/rules</code> file:</p>
<pre>
Ping(REJECT):info $FW net:^[US]
</pre>
<ul><li>The format for referencing countries is <code>^[<Country Code>]</code>.</li>
<li>Multiple countries can be specified in the brackets, e.g. <code>^[US,FR,CA,UK]</code>.</li>
<li>See http://shorewall.net/ISO-3661.html for more information.</li></ul>
<p>Check and restart Shorewall:</p>
<pre>
root@ubuntu:/etc/shorewall# shorewall check
root@ubuntu:/etc/shorewall# shorewall restart
</pre>
<p>Now we'll test ping to <code>www.google.com</code>:</p>
<pre>
root@ubuntu:/etc/shorewall# ping -n -c 2 www.google.com
PING www.google.com (64.233.185.105) 56(84) bytes of data.
From 192.168.1.100 icmp_seq=1 Destination Host Unreachable
From 192.168.1.100 icmp_seq=1 Destination Host Unreachable
--- www.google.com ping statistics ---
0 packets transmitted, 0 received, +2 errors
</pre>
<p>Working as intended. Let's see what Shorewall logged:</p>
<pre>
Feb 6 23:34:27 ubuntu kernel: [ 942.806263] Shorewall:fw2net:REJECT:IN= OUT=eth0 SRC=192.168.1.100 DST=64.233.185.104 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=9477 DF PROTO=ICMP TYPE=8 CODE=0 ID=5713 SEQ=1
Feb 6 23:34:27 ubuntu kernel: [ 942.807537] Shorewall:fw2net:REJECT:IN= OUT=eth0 SRC=192.168.1.100 DST=64.233.185.104 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=9478 DF PROTO=ICMP TYPE=8 CODE=0 ID=5713 SEQ=1
Feb 6 23:35:26 ubuntu kernel: [ 1002.101682] Shorewall:fw2net:REJECT:IN= OUT=eth0 SRC=192.168.1.100 DST=64.233.185.105 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=63477 DF PROTO=ICMP TYPE=8 CODE=0 ID=5730 SEQ=1
Feb 6 23:35:26 ubuntu kernel: [ 1002.101924] Shorewall:fw2net:REJECT:IN= OUT=eth0 SRC=192.168.1.100 DST=64.233.185.105 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=63478 DF PROTO=ICMP TYPE=8 CODE=0 ID=5730 SEQ=1
</pre>
<p>And for more verification, to see it isn't blocking IP addresses that are non-US owned, we'll ping <code>www.pcengines.ch</code>:</p>
<pre>
root@ubuntu:/etc/shorewall# ping -n -c 2 www.pcengines.ch
PING www.pcengines.ch (213.133.104.38) 56(84) bytes of data.
64 bytes from 213.133.104.38: icmp_seq=1 ttl=51 time=297 ms
64 bytes from 213.133.104.38: icmp_seq=2 ttl=51 time=296 ms
--- www.pcengines.ch ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 296.200/296.846/297.493/0.845 ms
</pre>
<h3><p>Conclusion</p></h3>
<p>Using GeoIP support in Shorewall is a much quicker way to blacklist <em>or</em> whitelist large country IP address ranges, when compared to using just ipsets. Though this does not diminish the use cases of ipsets, as they are very useful in many different solutions which I will eventually cover in the future.</p>
<h3>References</h3>
<ul>
<li><a href="https://bugs.launchpad.net/ubuntu/+source/xtables-addons/+bug/1286911">https://bugs.launchpad.net/ubuntu/+source/xtables-addons/+bug/1286911</a></li>
<li><a href="http://xtables-addons.sourceforge.net/geoip.php">http://xtables-addons.sourceforge.net/geoip.php</a></li>
<li><a href="http://shorewall.net/ISO-3661.html">http://shorewall.net/ISO-3661.html</a></li>
</ul>
Rodolf Sabalburohttp://www.blogger.com/profile/06648192806453188759noreply@blogger.com4tag:blogger.com,1999:blog-8509744489347868336.post-22502759977491528572014-07-25T01:28:00.001-07:002015-02-06T09:50:12.438-08:00fail2ban with Shorewall<p>When it came to blacklisting attackers trying to brute-force my services, like SSH, my go-to package has always been <a href="http://en.wikipedia.org/wiki/DenyHosts">DenyHosts</a>. However, <a href="http://askubuntu.com/questions/433924/package-denyhosts-in-ubuntu-trusty-tahr-is-deleted-temporary-or-forever">issues</a> such as recent vulnerabilities and most notably, its removal from the default repositories for Ubuntu 14.04 LTS caused me to finally switch to <code>fail2ban</code>. The biggest advantage <code>fail2ban</code> provides over DenyHosts is that it is more flexible in its actions and types of services it can monitor (DenyHosts only supports services using TCP wrappers).</p>
<h3>Installation</h3>
<p>Install the <code>fail2ban</code> package:</p>
<pre>
root@localhost:~# aptitude -yvV install fail2ban
</pre>
<p>After installation, the <code>fail2ban</code> service will automatically be started:</p>
<p><pre><code>
root@localhost:/etc/fail2ban# ps aux | grep [f]ail2ban
root 11232 0.0 1.8 268164 9360 ? Sl 02:47 0:00 /usr/bin/python /usr/bin/fail2ban-server -b -s /var/run/fail2ban/fail2ban.sock -p /var/run/fail2ban/fail2ban.pid
root@localhost:/etc/fail2ban# service fail2ban status
* Status of authentication failure monitor
* fail2ban is running
</code></pre></p>
<h3>Configuration</h3>
<p>First, make a copy of the <code>/etc/fail2ban/jail.conf</code> file as <code>/etc/fail2ban/jail.local</code>, as per <a href="http://www.fail2ban.org/wiki/index.php/MANUAL_0_8">http://www.fail2ban.org/wiki/index.php/MANUAL_0_8</a>:</p>
<blockquote>
Every .conf file can be overridden with a file named .local. The .conf file is read first, then .local, with later settings overriding earlier ones. Thus, a .local file doesn't have to include everything in the corresponding .conf file, only those settings that you wish to override.
Modifications should take place in the .local and not in the .conf. This avoids merging problem when upgrading.
</blockquote>
<p><pre>
root@localhost:/etc/fail2ban# cp -v jail.conf jail.local
‘jail.conf’ -> ‘jail.local’
</pre></p>
<p>By default <code>fail2ban</code> uses <code>iptables</code> to control its default banning actions. Luckily using Shorewall is as simple as changing a single line in the <code>/etc/fail2ban/jail.local</code> configuration file.</p>
<p><strong>/etc/fail2ban/jail.local</strong></p>
<pre>
#
# ACTIONS
#
# Default banning action (e.g. iptables, iptables-new,
# iptables-multiport, shorewall, etc) It is used to define
# action_* variables. Can be overridden globally or per
# section within jail.local file
#banaction = iptables-multiport
banaction = shorewall
</pre>
<p>The <a href="https://github.com/fail2ban/fail2ban/blob/master/config/action.d/shorewall.conf"><code>/etc/fail2ban/action.d/shorewall.conf</code></a> configuration file comments explains this in more detail.</p>
<p>All files in the <code>/etc/fail2ban/action.d/</code> drop configuration directory control actions to an accompanying service. For example, <code>action.d/ipfw.conf</code> controls how <code>fail2ban</code> tells <code><a href="https://en.wikipedia.org/wiki/Ipfw">ipfw</a></code>, used in FreeBSD systems, what command to run to ban an offending host. Likewise, <code>action.d/shorewall.conf</code> controls what commands are run by <code>shorewall</code> in order to ban an offender.</p>
<p>In <code>/etc/fail2ban/action.d/shorewall.conf</code> the important directives to pay attention to for our setup are <code>actionban</code>, <code>actionunban</code>, and <code>blocktype</code>:</p>
<pre>
# Option: actionban
# Notes.: command executed when banning an IP. Take care that the
# command is executed with Fail2Ban user rights.
# Tags: See jail.conf(5) man page
# Values: CMD
#
actionban = shorewall <blocktype> <ip>
# Option: actionunban
# Notes.: command executed when unbanning an IP. Take care that the
# command is executed with Fail2Ban user rights.
# Tags: See jail.conf(5) man page
# Values: CMD
#
actionunban = shorewall allow <ip>
[Init]
# Option: blocktype
# Note: This is what the action does with rules.
# See man page of shorewall for options that include drop, logdrop, reject, or logreject
# Values: STRING
#blocktype = reject
blocktype = drop
</pre>
<ul>
<li>To reiterate, when <code>fail2ban</code> bans a host, the command <code>shorewall drop <offending IP address></code> will be run.</li>
</ul>
<p>In the above configuration I changed the <code>blocktype</code> to <code>drop</code> rather than <code>reject</code>. Rather than getting an ICMP destination unreachable response from my server when attackers are brute-forcing my services like SSH after they have been banned, <code>drop</code> will cause their connections to timeout instead.</p>
<p>Also, <code>shorewall</code> needs to be updated to handle the banning action correctly. By default, it will only blacklist hosts part of a new TCP connection, but not already existing or established connections.</p>
<p><strong>/etc/shorewall/shorewall.conf</strong></p>
<pre>
#BLACKLIST="NEW,INVALID,UNTRACKED"
BLACKLIST="ALL"
</pre>
<p>Restart the <code>fail2ban</code> and <code>shorewall</code> services:</p>
<pre>
root@localhost:/etc/fail2ban# service fail2ban restart
* Restarting authentication failure monitor fail2ban [ OK ]
root@localhost:/etc/shorewall# service shorewall restart
Restarting "Shorewall firewall": done.
</pre>
<p></p>
<h3>Seeing it in action</h3>
<p>After leaving <code>fail2ban</code> running for a couple hours, you are inevitably going to get some banned hosts. You can see that <code>fail2ban</code> is working correctly by checking the <code>/var/log/fail2ban.log</code> log file. Here's an example of some IPs that were banned:</p>
<pre>
2014-07-23 19:58:16,068 fail2ban.actions: WARNING [ssh] Ban 111.74.238.167
2014-07-23 20:08:16,818 fail2ban.actions: WARNING [ssh] Unban 111.74.238.167
2014-07-23 20:46:02,506 fail2ban.actions: WARNING [ssh] Ban 117.21.225.116
2014-07-23 20:56:03,249 fail2ban.actions: WARNING [ssh] Unban 117.21.225.116
</pre>
<ul>
<li>Note the hosts were unbanned after 10 minutes, which is the default timer.</li>
<li>To increase the ban time, change the <code>bantime</code> value in the <code>/etc/fail2ban/jail.local</code> configuration file.</li>
</ul>
<p>To verify that <code>fail2ban</code> did indeed pass the IPs to <code>shorewall</code> use the <code>shorewall show dynamic</code> command:</p>
<p>
<pre>
root@localhost:/etc/fail2ban# shorewall show dynamic
Shorewall 4.5.21.6 Chain dynamic at localhost - Fri Jul 25 07:21:26 EDT 2014
Counters reset Wed Jul 23 03:07:05 EDT 2014
Chain dynamic (4 references)
pkts bytes target prot opt in out source destination
0 0 DROP all -- * * 183.56.129.146 0.0.0.0/0
0 0 DROP all -- * * 54.210.100.16 0.0.0.0/0
0 0 DROP all -- * * 115.239.248.61 0.0.0.0/0
0 0 DROP all -- * * 61.147.103.71 0.0.0.0/0
0 0 DROP all -- * * 115.239.248.50 0.0.0.0/0
0 0 DROP all -- * * 117.21.191.209 0.0.0.0/0
0 0 DROP all -- * * 117.21.191.210 0.0.0.0/0
0 0 DROP all -- * * 110.12.207.5 0.0.0.0/0
</pre>
</p>
<h3>Resources</h3>
<p>
https://www.digitalocean.com/community/tutorials/how-to-protect-ssh-with-fail2ban-on-ubuntu-12-04<br />
http://www.ehow.com/how_12187342_fail2ban-shorewall.html
</p>Rodolf Sabalburohttp://www.blogger.com/profile/06648192806453188759noreply@blogger.com3tag:blogger.com,1999:blog-8509744489347868336.post-79556850345214749282014-07-13T05:39:00.001-07:002014-07-13T05:39:23.595-07:00Redirecting Shorewall log messages to custom file<p>By default Shorewall will log firewall messages to <code>/var/log/messages</code> if you are running CentOS, or <code>/var/log/syslog</code> if you are running Ubuntu.</p>
<p>Sample log message from Shorewall using default format:</p>
<pre>
Jul 13 01:30:42 localhost kernel: [37078.720986] Shorewall:net2fw:DROP:IN=eth0 OUT= MAC=00:0c:29:85:73:f2:00:1d:d0:cd:3b:81:08:00 SRC=128.72.226.39 DST=192.168.0.16 LEN=129 TOS=0x08 PREC=0x20 TTL=104 ID=19418 PROTO=UDP SPT=11888 DPT=51413 LEN=109
</pre>
<p>Despite the <code>LOGFILE=/var/log/messages</code> parameter in the <code>shorewall.conf</code> file, this directive isn't used to redirect logs to a custom file. As described from <code>man shorewall.conf</code>:</p>
<pre>
LOGFILE=[pathname]
This parameter tells the /sbin/shorewall program where to look for Shorewall messages
when processing the dump, logwatch, show log, and hits commands. If not assigned or if
assigned an empty value, /var/log/messages is assumed. For further information, see
http://www.shorewall.net/shorewall_logging.html.
</pre>
<p></p>
<h3>Using rsyslog to redirect messages</h3>
<p>Configurations done through <code>rsyslog</code> will allow Shorewall log messages to be redirected to a custom file.</p>
<p>Create the <code>/etc/rsyslog.d/40-shorewall.conf</code> file:</p>
<pre>
# touch /etc/rsyslog.d/40-shorewall.conf
# ls -l /etc/rsyslog.d
total 8
-rw-r--r-- 1 root root 311 Mar 17 2012 20-ufw.conf
-rw-r--r-- 1 root root 0 Jul 13 02:04 40-shorewall.conf
-rw-r--r-- 1 root root 1655 May 19 2013 50-default.conf
</pre>
<ul>
<li>The numerical prefix determines the order at which the configuration files are read by <code>rsyslog</code>, i.e. settings in <code>40-shorewall.conf</code> will be configured before <code>50-default.conf</code>.</li>
</ul>
<p>Edit <code>/etc/rsyslog.d/40-shorewall.conf</code>:</p>
<pre>
:msg, contains, "Shorewall:" -/var/log/shorewall.log
& ~
</pre>
<ul>
<li>This is an <code>rsyslog</code> <a href="http://www.rsyslog.com/doc/rsyslog_conf_filter.html">property-based filter</a> which matches any log messages which contain the string <em>Shorewall:</em> and redirects them to the <code>/var/log/shorewall.log</code> log file.</li>
<li>The "<code>& ~</code>" tells <code>rsyslog</code> to stop processing the message after it was written to the log. Without the "<code>& ~</code>", messages would continue to be written to local files such as <code>/var/log/syslog</code>.</li>
<ul>
<li>If you wanted messages to still show up in <code>/var/log/{syslog,messages}</code> then the "<code>& ~</code>" can be omitted.</li>
</ul>
</ul>
<p>Anytime custom log files are made, an accompanying <code>/etc/logrotate.d</code> configuration file should be created or updated. Shorewall by default creates <code>/etc/logrotate.d/shorewall</code> which controls rotation of the <code>/var/log/shorewall-init.log</code> file. Add a directive to control <code>/var/log/shorewall.log</code>:</p>
<pre>
# cat /etc/logrotate.d/shorewall
/var/log/shorewall-init.log {
weekly
rotate 4
compress
missingok
create 0640 root adm
}
/var/log/shorewall.log {
weekly
rotate 52
compress
compressext .gz
missingok
create 0640 root adm
}
</pre>
<ul>
<li>This will rotate the <code>shorewall.log</code> log file every week and store archived log files for 52 weeks (1 year).</li>
</ul>
<p>Restart the <code>rsyslog</code> service:</p>
<pre>
# service rsyslog restart
</pre>
<p>The restart should have triggered the <code>/var/log/shorewall.log</code> file to be created, if not, try to port-scan or trigger an event and see if the file was created:</p>
<pre>
# ls -l /var/log/shorewall.log
-rw-r----- 1 syslog adm 0 Jul 13 02:27 /var/log/shorewall.log
</pre>
<p></p>
<h3>rsyslog template</h3>
<p>I created a template to shorten the log messages, as Shorewall log messages can be quite long and verbose, by adding another line to <code>/etc/rsyslog.d/40-shorewall.conf</code>:</p>
<pre>
$template shorewall-template,"%timegenerated% %msg%\n"
:msg, contains, "Shorewall:" -/var/log/shorewall.log;shorewall-template
& ~
</pre>
<ul>
<li><code>rsyslog</code> filtering does not follow a simple syntax, so I recommend reading their <a href="http://www.rsyslog.com/doc/master/index.html">official documentation</a> to learn exactly how it works, as <code>rsyslog</code> can be leveraged to do very complex and powerful configurations.</li>
<li>Though not much, the template removes the <code>localhost kernel:</code> portion in the message:</li>
<pre>
From this:
Jul 13 01:30:42 localhost kernel: [37078.720986] Shorewall:net2fw:DROP:IN=eth0 OUT= MAC=00:0c:29:85:73:f2:00:1d:d0:cd:3b:81:08:00 SRC=128.72.226.39 DST=192.168.0.16 LEN=129 TOS=0x08 PREC=0x20 TTL=104 ID=19418 PROTO=UDP SPT=11888 DPT=51413 LEN=109
To this:
Jul 13 01:30:42 [37078.720986] Shorewall:net2fw:DROP:IN=eth0 OUT= MAC=00:0c:29:85:73:f2:00:1d:d0:cd:3b:81:08:00 SRC=128.72.226.39 DST=192.168.0.16 LEN=129 TOS=0x08 PREC=0x20 TTL=104 ID=19418 PROTO=UDP SPT=11888 DPT=51413 LEN=109
</pre>
<li>Full list of properties available <a href="http://www.rsyslog.com/doc/property_replacer.html">here</a>.</li>
</ul>
Rodolf Sabalburohttp://www.blogger.com/profile/06648192806453188759noreply@blogger.com6tag:blogger.com,1999:blog-8509744489347868336.post-64522743768352349532014-07-09T05:40:00.000-07:002014-07-12T19:23:13.844-07:00Initial network analysis of stray BitTorrent traffic<p>In my <a href="http://rsabalburo.blogspot.com/2014/07/ntopng-on-ubuntu-1404-lts-server.html">previous</a> blog post I detailed a basic installation of <code>ntopng</code>. Interestingly, I was able to detect odd traffic destined to one of my desktops in my network ‒ no less than a day after I had setup <code>ntopng</code> on a development server. Network forensics and analysis is a domain I've been meaning to level up, so this was a perfect <strike>opportunity</strike> quest to explore.</p>
<p>In the <code>ntopng</code> web interface under the <em>Flows</em> section, <code>ntopng</code> had identified an active flow with a duration of over <em>4h, 14 min</em> originating from the IP address <code>82.207.24.165</code>, which resides in Kiev, Ukraine. There was a nice little <a href="https://duckduckgo.com/?t=lm&q=ukraine+flag">Ukraine flag</a> by the IP in the web interface, so it quickly caught my eye. In addition, the flow was identified as the <a href="http://en.wikipedia.org/wiki/BitTorrent">BitTorrent</a> protocol.</p>
<div class="separator" style="clear: both; text-align: center;"><a href="http://1.bp.blogspot.com/-0nPDrwQ8VmU/U70d3Qq2ltI/AAAAAAAAAN0/9xoeXKkUajY/s1600/af.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://1.bp.blogspot.com/-0nPDrwQ8VmU/U70d3Qq2ltI/AAAAAAAAAN0/9xoeXKkUajY/s640/af.png" /></a><figcaption>Example screenshot of <em>Active Flows</em> section inside web interface</figcaption></div>
<p>Foreign IP addresses are to be expected, especially in regards to the BitTorrent protocol. The problem, however, was that my desktop wasn't running a BitTorrent client at the time and BitTorrent almost always connects to several peers at once for both (sometimes simultaneously) uploads and downloads. So my immediate questions were how was this traffic getting through the router firewall and why was it only this single IP address? Seeing a bunch of foreign flags in the web interface at once would have actually worried me less.</p>
<p>Since the flow was still active, I decided to see what it looked like on the wire with <code>tcpdump</code> by using a <a href="https://en.wikipedia.org/wiki/Berkeley_Packet_Filter">BPF</a> to match any traffic to or from that Ukraine host:</p>
<pre>
root@trustytahr:/var/tmp# tcpdump -v -n -i eth0 not tcp port 22 and ip host 82.207.24.165
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
13:14:49.233184 IP (tos 0x0, ttl 107, id 5434, offset 0, flags [none], proto UDP (17), length 129)
82.207.24.165.6881 > 192.168.0.7.31369: UDP, length 101
13:14:49.356179 IP (tos 0x0, ttl 107, id 5449, offset 0, flags [none], proto UDP (17), length 129)
82.207.24.165.6881 > 192.168.0.7.31369: UDP, length 101
13:14:49.438760 IP (tos 0x0, ttl 107, id 5460, offset 0, flags [none], proto UDP (17), length 129)
82.207.24.165.6881 > 192.168.0.7.31369: UDP, length 101
13:14:49.544598 IP (tos 0x0, ttl 107, id 5470, offset 0, flags [none], proto UDP (17), length 129)
82.207.24.165.6881 > 192.168.0.7.31369: UDP, length 101
13:14:49.583587 IP (tos 0x0, ttl 107, id 5475, offset 0, flags [none], proto UDP (17), length 129)
82.207.24.165.6881 > 192.168.0.7.31369: UDP, length 101
13:14:49.735788 IP (tos 0x0, ttl 107, id 5489, offset 0, flags [none], proto UDP (17), length 129)
82.207.24.165.6881 > 192.168.0.7.31369: UDP, length 101
13:14:49.804994 IP (tos 0x0, ttl 107, id 5498, offset 0, flags [none], proto UDP (17), length 129)
82.207.24.165.6881 > 192.168.0.7.31369: UDP, length 101
13:14:49.839829 IP (tos 0x0, ttl 107, id 5502, offset 0, flags [none], proto UDP (17), length 129)
82.207.24.165.6881 > 192.168.0.7.31369: UDP, length 101
13:14:49.926611 IP (tos 0x0, ttl 107, id 5512, offset 0, flags [none], proto UDP (17), length 129)
82.207.24.165.6881 > 192.168.0.7.31369: UDP, length 101
13:14:49.965995 IP (tos 0x0, ttl 107, id 5516, offset 0, flags [none], proto UDP (17), length 129)
82.207.24.165.6881 > 192.168.0.7.31369: UDP, length 101
</pre>
<ul>
<li>UDP packets were being sent several times a second by <code>82.207.24.165</code> with random source ports.</li>
<li>All of these packets were being sent to the same UDP port, <code>31368</code>, on <code>192.168.0.7</code>.</li>
<li><code>-v</code> specifies verbose mode, and <code>-n</code> prevents <code>tcpdump</code> from doing a reverse DNS lookup on the IP addresses.</li>
</ul>
<p>Seeing that the traffic was identified as BitTorrent and destined to <code>192.168.0.7:31369</code>, I checked three things on <code>192.168.0.7</code> (a desktop running Windows 7):</p>
<ol>
<li>Find any connections matching port <code>31369</code>, and also see if there were any interesting or malicious processes using the port.</li>
<pre>
C:\Windows\system32>netstat -nabo | findstr "31369"
C:\Windows\system32>
</pre>
<ul><li>No connections found.</li></ul>
<li>Double-check if µTorrent was running. A quick check in the system tray and Windows Task Manager verified it wasn't.</li>
<li>See what random port µTorrent was using; µTorrent assigns itself a random UDP port for incoming connections.</li>
<br />
<div class="separator" style="clear: both; text-align: center;"><a href="http://2.bp.blogspot.com/-86OWYARruTY/U70kYXi0IzI/AAAAAAAAAOA/NXO9jsGbEb8/s1600/bt.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://2.bp.blogspot.com/-86OWYARruTY/U70kYXi0IzI/AAAAAAAAAOA/NXO9jsGbEb8/s640/bt.png" /></a><figcaption><code>31389</code> is indeed the randomly assigned port for incoming connections<br />Also take note how <em>Enable UPnP port mapping</em> and <em>Enable NAT-PMP port mapping</em> were enabled</figcaption></div>
</ol>
<p>For further exploration, I re-ran <code>tcpdump</code> and included the <code>-w</code> switch which outputs the captured network traffic to a named <code>.pcap</code> file. I then opened it in Wireshark and clicked <em>Follow UDP Stream</em>.</p>
<div class="separator" style="clear: both; text-align: center;"><a href="http://1.bp.blogspot.com/-wBXSwt8Vu7I/U70NA136yaI/AAAAAAAAANc/gyv-o4NRCYY/s1600/u.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://1.bp.blogspot.com/-wBXSwt8Vu7I/U70NA136yaI/AAAAAAAAANc/gyv-o4NRCYY/s640/u.png" /></a><figcaption>Screenshot of <em>Follow UDP Stream</em> in Wireshark</figcaption></div>
<p>Searching Google for strings in the output lead me to <a href="http://www.bittorrent.org/beps/bep_0005.html">For Developers</a> section on the <a href="http://www.bittorrent.org">BitTorrent.org</a> website (not to be used with <a href="http://www.bittorrent.org">BitTorrent.com</a>). On the page it describes the <em>find_node</em> packets:</p>
<blockquote>
Find node is used to find the contact information for a node given its ID. "q" == "find_node" A find_node query has two arguments, "id" containing the node ID of the querying node, and "target" containing the ID of the node sought by the queryer. When a node receives a find_node query, it should respond with a key "nodes" and value of a string containing the compact node info for the target node or the K (8) closest good nodes in its own routing table.
</blockquote>
<p>Also, it shows an example packet for a <em>find_node Query</em>, which is the type of packet that I was receiving from the Ukraine host:</p>
<pre>
find_node Query = {"t":"aa", "y":"q", "q":"find_node", "a": {"id":"abcdefghij0123456789", "target":"mnopqrstuvwxyz123456"}}
bencoded = d1:ad2:id20:abcdefghij01234567896:target20:mnopqrstuvwxyz123456e1:q9:find_node1:t2:aa1:y1:qe
</pre>
<p>Alternatively, the <em>find_node Response</em> packet looks like so (though my desktop never replied to a query in the time it was an active flow listed in <code>ntopng</code>):</p>
<pre>
Response = {"t":"aa", "y":"r", "r": {"id":"0123456789abcdefghij", "nodes": "def456..."}}
bencoded = d1:rd2:id20:0123456789abcdefghij5:nodes9:def456...e1:t2:aa1:y1:re
</pre>
<p></p>
<h3>UPnP, NAT-PMP and BitTorrent</h3>
<blockquote>
UPnP stands for “Universal Plug and Play.” Using UPnP, an application can automatically forward a port on your router, saving you the hassle of forwarding ports manually. We’ll be looking at the reasons people recommend disabling UPnP, so we can get a clear picture of the security risks. — <a href="http://www.howtogeek.com/122487/htg-explains-is-upnp-a-security-risk/">http://www.howtogeek.com/122487/htg-explains-is-upnp-a-security-risk/</a></blockquote>
<blockquote>NAT-PMP allows a computer in a private network (behind a NAT router) to automatically configure the router to allow parties outside the private network to contact it. NAT-PMP runs over UDP port 5351. It essentially automates the process of port forwarding. — <a href="https://en.wikipedia.org/wiki/NAT-PMP">Wikipedia</a></blockquote>
<p>UPnP and NAT-PMP, though different protocols, essentially do the same thing. UPnP or NAT-PMP run as a service on many home-based routers, and is usually enabled by default. BitTorrent often relies on these protocols to allow traffic through networks behind a NAT, which is nearly every home network. All of the major BitTorrent <a href="https://en.wikipedia.org/wiki/NAT-PMP#Applications_support">applications support</a> support UPnP and NAT-PMP, which allows these clients to map ports on the router.</p>
<p>Turning off UPnP or NAT-PMP is as simple as going into the router web interface and looking for the checkbox to disable it.</p>
<h3>Monitoring</h3>
<p>Though the flow eventually stopped after a few hours, before I could test to see if turning off UPnP and NAT-PMP made a difference, I've been using <code>ntopng</code> to continually monitor and analyze other anomalous traffic.</p>
Rodolf Sabalburohttp://www.blogger.com/profile/06648192806453188759noreply@blogger.com0tag:blogger.com,1999:blog-8509744489347868336.post-50242869148118143192014-07-06T06:00:00.001-07:002015-02-15T12:29:05.537-08:00ntopng on Ubuntu 14.04 LTS Server<p><strong>Following directions are deprecated, updated instructions are available at <a href="http://rsabalburo.blogspot.ca/2015/02/ntopng-122-on-ubuntu-1404-revisited.html">ntopng 1.2.2 on Ubuntu 14.04, revisited</a>.</strong></p>
<p><hr /></p>
<blockquote>
ntopng is the next generation version of the original ntop, a network traffic probe that shows the network usage, similar to what the popular top Unix command does. ntopng is based on libpcap and it has been written in a portable way in order to virtually run on every Unix platform, MacOSX and on Win32 as well. — <a href="http://www.ntop.org/products/ntop/">http://www.ntop.org/products/ntop/</a>
</blockquote>
<p><code>ntopng</code> is different from the original <code>ntop</code> package found in the default Ubuntu repositories. Perhaps the biggest different between the two is that <code>ntopng</code> has an updated and modern HTML5 web interface which has data visualizations that update in real-time, whereas <code>ntop</code> provides a static and very generic looking web interface.</p>
<h3>Installing ntopng</h3>
<p><em>Note that packages from the repository below are x64 only. If you are using i386, then ntopng must be built from <a href="http://sourceforge.net/projects/ntop/files/ntopng/">source</a>.</em></p>
<p>Following the download pages on ntop.org and looking for stable Ubuntu packages will eventually lead you to <a href="http://www.nmon.net/apt-stable/">this</a> website repository. The directions below follow those in the listed website.</p>
<p>Add the http://www.nmon.net/apt-stable/ repository and its accompanying public key:</p>
<pre>
ubuntu@trustytahr:~$ sudo -i
[sudo] password for ubuntu:
root@trustytahr:~# /bin/echo -e "deb http://www.nmon.net/apt-stable/ x64/\ndeb http://www.nmon.net/apt-stable/ all/" > /etc/apt/sources.list.d/ntop.list
root@trustytahr:~# wget -qO - http://www.nmon.net/apt-stable/ntop.key | sudo apt-key add -
OK
</pre>
<ul>
<li>Switch completely to the root user, rather than using <code>sudo</code> to run the above commands.</li>
</ul>
<p>Update <code>apt</code> repositories with <code>apt-get update</code> and ignore the conflicting messages at the end:</p>
<pre>
root@trustytahr:~# apt-get update
Reading package lists... Done
W: Conflicting distribution: http://www.nmon.net x64/ Release (expected x64 but got )
W: Conflicting distribution: http://www.nmon.net all/ Release (expected all but got )
</pre>
<p>Install the <code>ntopng</code> and <code>ntopng-data</code> packages:</p>
<pre>
root@trustytahr:~# apt-get -y install ntopng ntopng-data
</pre>
<p>The installation creates startup script <code>/etc/init.d/ntopng</code>, and the <code>ntopng</code> binary itself as <code>/usr/local/bin/ntopng</code>. The rest of the relevant files can be all found under the <code>/usr/local/share/ntopng</code> directory. See output of <em><code>dpkg -L ntopng</code></em> for a full listing.</p>
<h3>Running ntopng</h3>
<p>Starting <code>ntopng</code> is as simple as executing the <code>/usr/local/bin/ntopng</code> binary:</p>
<pre>
root@trustytahr:/usr/local/bin# ./ntopng
06/Jul/2014 01:15:34 [Ntop.cpp:555] Setting local networks to 192.168.1.0/24,0.0.0.0/32,224.0.0.0/8,239.0.0.0/8,255.255.255.255/32,127.0.0.0/8
06/Jul/2014 01:15:34 [Redis.cpp:50] Successfully connected to Redis 127.0.0.1:6379
06/Jul/2014 01:15:34 [PcapInterface.cpp:81] Reading packets from interface eth0...
06/Jul/2014 01:15:34 [Ntop.cpp:662] Registered interface eth0 [id: 0]
06/Jul/2014 01:15:34 [PcapInterface.cpp:81] Reading packets from interface lo...
06/Jul/2014 01:15:34 [Ntop.cpp:662] Registered interface lo [id: 1]
06/Jul/2014 01:15:34 [Utils.cpp:251] User changed to nobody
06/Jul/2014 01:15:34 [main.cpp:152] PID stored in file /var/tmp/ntopng.pid
06/Jul/2014 01:15:34 [HTTPserver.cpp:351] HTTPS Disabled: missing SSL certificate /usr/local/share/ntopng/httpdocs/ssl/ntopng-cert.pem
06/Jul/2014 01:15:34 [HTTPserver.cpp:352] Please read README.SSL if you want to enable SSL
06/Jul/2014 01:15:34 [HTTPserver.cpp:389] Web server dirs [/usr/local/share/ntopng/httpdocs][/usr/local/share/ntopng/scripts]
06/Jul/2014 01:15:34 [HTTPserver.cpp:392] HTTP server listening on port 3000
06/Jul/2014 01:15:34 [main.cpp:186] Using RRD version 1.4.7
06/Jul/2014 01:15:34 [main.cpp:202] Working directory: /var/tmp/ntopng
06/Jul/2014 01:15:34 [main.cpp:204] Scripts/HTML pages directory: /usr/local/share/ntopng
06/Jul/2014 01:15:34 [Ntop.cpp:181] Welcome to ntopng x86_64 v.1.1.4 (r7806) - (C) 1998-14 ntop.org
06/Jul/2014 01:15:34 [PeriodicActivities.cpp:53] Started periodic activities loop...
06/Jul/2014 01:15:34 [NetworkInterface.cpp:770] Started packet polling on interface eth0...
06/Jul/2014 01:15:34 [NetworkInterface.cpp:770] Started packet polling on interface lo...
</pre>
<ul>
<li>Above output shows <code>./ntopng</code> run as a foreground process, optionally it could have been ran as a background process with <code>./ntopng &</code>.</li>
<li>By default the working directory is <code>/var/tmp</code> and the listening port for the web interface is at port 3000.</p>
</ul>
<p>While the process is running, the web interface should now be accessible via browser at http://<SERVER IP>:3000 — http://localhost:3000 if viewing the web page locally.</p>
<p>The default username is <em>admin</em> and password is <em>admin</em>.</p>
<h3>Running ntopng as a service</h3>
<p>Running <code>ntopng</code> as a service, rather than a foreground or background script requires additional steps.</p>
<p>Note that <code>ntopng</code> has several options that can be passed at either the command-line or via configuration file. All the options are listed in the <code>man</code> page.</p>
<p>A configuration file should be created at <code>/etc/ntopng/ntopng.conf</code>, below is a configuration file I used (the commented descriptions are excerpts directly taken from <code>man ntopng</code>):</p>
<pre>
# /etc/ntopng/ntopng.conf
#
# The configuration file is similar to the command line, with the exception that an equal
# sign '=' must be used between key and value. Example: -i=p1p2 or --interface=p1p2 For
# options with no value (e.g. -v) the equal is also necessary. Example: "-v=" must be used.
#
#
# -G|--pid-path
# Specifies the path where the PID (process ID) is saved.
#
-G=/var/tmp/ntopng.pid
#
# -e|--daemon
# This parameter causes ntop to become a daemon, i.e. a task which runs in the background
# without connection to a specific terminal. To use ntop other than as a casual monitoring
# tool, you probably will want to use this option.
#
-e=
#
# -i|--interface
# Specifies the network interface or collector endpoint to be used by ntopng for network
# monitoring. On Unix you can specify both the interface name (e.g. lo) or the numeric
# interface id as shown by ntopng -h. On Windows you must use the interface number instead.
# Note that you can specify -i multiple times in order to instruct ntopng to create multi‐
# ple interfaces.
#
-i=eth0
#
# -w|--http-port
# Sets the HTTP port of the embedded web server.
#
-w=3000
#
# -m|--local-networks
# ntopng determines the ip addresses and netmasks for each active interface. Any traffic on
# those networks is considered local. This parameter allows the user to define additional
# networks and subnetworks whose traffic is also considered local in ntopng reports. All
# other hosts are considered remote. If not specified the default is set to 192.168.1.0/24.
#
# Commas separate multiple network values. Both netmask and CIDR notation may be used,
# even mixed together, for instance "131.114.21.0/24,10.0.0.0/255.0.0.0".
#
-m=192.168.1.0/24
#
# -n|--dns-mode
# Sets the DNS address resolution mode: 0 - Decode DNS responses and resolve only local
# (-m) numeric IPs 1 - Decode DNS responses and resolve all numeric IPs 2 - Decode DNS
# responses and don't resolve numeric IPs 3 - Don't decode DNS responses and don't resolve
#
-n=1
#
# -S|--sticky-hosts
# ntopng periodically purges idle hosts. With this option you can modify this behaviour by
# telling ntopng not to purge the hosts specified by -S. This parameter requires an argu‐
# ment that can be "all" (Keep all hosts in memory), "local" (Keep only local hosts),
# "remote" (Keep only remote hosts), "none" (Flush hosts when idle).
#
-S=
#
# -d|--data-dir
# Specifies the data directory (it must be writable). Default directory is ./data
#
-d=/var/tmp/ntopng
#
# -q|--disable-autologout
# Disable web interface logout for inactivity.
#
-q=
</pre>
<p>Create the <code>/etc/ntopng/ntopng.start</code> as an empty file:</p>
<pre>
root@trustytahr:~# touch /etc/ntopng/ntopng.start
root@trustytahr:~# ls -l /etc/ntopng/ntopng.start
-rw-r--r-- 1 root root 0 Jul 6 01:41 /etc/ntopng/ntopng.start
</pre>
<ul>
<li>For some reason the <code>start_ntopng()</code> function in the <code>/etc/init.d/ntopng</code> startup script requires this file in order for the service to start.</li>
</ul>
<p>Start the <code>ntopng</code> service:</p>
<pre>
root@trustytahr:~# service ntopng start
Starting ntopng
</pre>
<p>The <code>ntopng</code> process can be verified with the <code>service ntopng status</code> or in <code>ps aux</code> output:</p>
<pre>
root@trustytahr:~# ps aux | grep [n]topng
nobody 2343 0.6 1.3 764432 27900 ? Ssl 01:45 0:00 /usr/local/bin/ntopng /etc/ntopng/ntopng.conf
root@trustytahr:~# service ntopng status
ntopng running as 2343
</pre>
<p></p>
<h3>Changing default password for web interface</h3>
<p><strong>Command-line Method</strong></p>
<p>Verify connectivity to <code>redis-server</code> with <code>redis-cli ping</code> command:</p>
<pre>
root@trustytahr:~# redis-cli ping
PONG
</pre>
<p>Generate an md5 hash of our new password with <code>md5sum</code>:</p>
<pre>
root@trustytahr:~# echo -n "NewPassword_4ntopng" | md5sum
a13a83f9a23740e9ba453b2df3c41f9a -
</pre>
<p>Use this hash in the <code>redis-cli SET user.admin.password</code> command:</p>
<pre>
root@trustytahr:~# redis-cli SET user.admin.password a13a83f9a23740e9ba453b2df3c41f9a
OK
</pre>
<p><strong>Web Interface Method</strong></p>
<p>After logging into the web interface:</p>
<p><em>Settings (Gear Icon) > Manage Users > Set Password</em></p>
<p></p>
<h3>Troubleshooting</h3>
<ul>
<li>If the service fails to start, run <code>/usr/local/bin/ntopng /etc/ntopng/ntopng.conf</code> directly to see all messages on <code>stdout</code>.</li>
<li>The log file can be found at: <code>/var/tmp/ntopng/ntopng.log</code></li>
<li><code>ntopng</code> requires the <code>redis-server</code> service to be running:</li>
<pre>
root@trustytahr:~# service redis-server status
redis-server is running
root@trustytahr:~# netstat -tunlp | grep -i redis-server
tcp 0 0 127.0.0.1:6379 0.0.0.0:* LISTEN 2139/redis-server 1
</pre>
</ul>
<p></p>
<h3>Bug Fixes</h3>
<p>The <code>/etc/init.d/ntopng</code> startup script has a few minor bugs that could use a couple fixes:</p>
<ul>
<li><code>/usr/local/bin/ntopng</code> can actually be run without the root user; however, the <code>ntopng</code> process will not be able to enter the listening interface into promiscuous mode and will still return a successful exit status.</li>
<li>The startup script parses for <code>PID_FILE</code> using <code>grep</code> to find the <code>-G</code> switch, but doesn't alternatively search for its synonymous long switch <code>--pid-file</code>.</li>
<li>The <code>service ntopng status</code> command will return nothing if <code>ntopng</code> hasn't started.</li>
</ul>
<p><code>diff -u</code> output for the above fixes:</p>
<pre>
root@trustytahr:~# diff -u /etc/init.d/ntopng.orig /etc/init.d/ntopng
--- /etc/init.d/ntopng.orig 2014-07-06 02:30:21.133552653 -1000
+++ /etc/init.d/ntopng 2014-07-06 02:44:55.057578340 -1000
@@ -13,6 +13,11 @@
# Short-Description: Start/stop ntopng web
### END INIT INFO
+if [ $(id -u) -ne 0 ]; then
+ echo "ntopng requires root."
+ exit 1
+fi
+
start_ntopng() {
FORCE=$1
@@ -34,7 +39,7 @@
if [ -f /etc/ntopng/ntopng.start ] || [ $FORCE -eq 1 ]; then
echo "Starting ntopng"
- PID_FILE=$(cat /etc/ntopng/ntopng.conf | grep '\-G='|cut -d '=' -f 2)
+ PID_FILE=$(grep '^-G\|^--pid-file' /etc/ntopng/ntopng.conf | cut -d '=' -f 2)
if [ -f $PID_FILE ]; then
PID=$(cat $PID_FILE)
if [ $PID -gt 0 ]; then
@@ -60,7 +65,7 @@
fi
if [ -f /etc/ntopng/ntopng.conf ]; then
- PID_FILE=$(cat /etc/ntopng/ntopng.conf | grep '\-G='|cut -d '=' -f 2)
+ PID_FILE=$(grep '^-G\|^--pid-file' /etc/ntopng/ntopng.conf | cut -d '=' -f 2)
if [ -f "$PID_FILE" ]; then
PID=$(cat $PID_FILE)
if [ $PID -gt 0 ]; then
@@ -86,7 +91,7 @@
return 0
fi
- PID_FILE=$(cat /etc/ntopng/ntopng.conf | grep '\-G='|cut -d '=' -f 2)
+ PID_FILE=$(grep '^-G\|^--pid-file' /etc/ntopng/ntopng.conf | cut -d '=' -f 2)
if [ -f $PID_FILE ]; then
PID=$(cat $PID_FILE)
if [ $PID -gt 0 ]; then
@@ -94,6 +99,8 @@
else
echo "No running ntopng pid [$PID] in [$PID_FILE]"
fi
+ else
+ echo "ntopng not running: process id file $PID_FILE not found."
fi
return 0
</pre>
<p></p>
<h3>References</h3>
<p>
http://xmodulo.com/2013/10/set-web-based-network-traffic-monitoring-linux.html<br>
http://blog.redbranch.net/2013/12/12/reset-ntopng-admin-password/
</p>Rodolf Sabalburohttp://www.blogger.com/profile/06648192806453188759noreply@blogger.com2tag:blogger.com,1999:blog-8509744489347868336.post-29566767972274079172014-07-04T04:11:00.001-07:002014-07-04T04:19:34.899-07:00SSD tweaks under Ubuntu 14.04 LTS Server<h3>Change IO Scheduler</h3>
<blockquote>
"The scheduler helps organise reads and writes in the I/O queue to maximise performance. The default scheduler in the Linux kernel is CFQ (Completely Fair Queuing) which is designed with the rotational latencies of spinning platter drives in mind. So while it works well for standard hard drives it doesn’t work so well when it comes to SSDs." — <a href="http://apcmag.com/how-to-maximise-ssd-performance-with-linux.htm">http://apcmag.com/how-to-maximise-ssd-performance-with-linux.htm</a>
</blockquote>
<p>Edit <code>/etc/default/grub</code> and add <code>elevator=deadline</code> to <code>GRUB_CMDLINE_LINUX_DEFAULT</code>:</p>
<pre>
#GRUB_CMDLINE_LINUX_DEFAULT=""
GRUB_CMDLINE_LINUX_DEFAULT="elevator=deadline"
</pre>
<p>Update GRUB settings with <code>update-grub2</code>:</p>
<pre>
root@localhost:~# update-grub2
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-3.13.0-30-generic
Found initrd image: /boot/initrd.img-3.13.0-30-generic
Found linux image: /boot/vmlinuz-3.13.0-24-generic
Found initrd image: /boot/initrd.img-3.13.0-24-generic
Found memtest86+ image: /memtest86+.elf
Found memtest86+ image: /memtest86+.bin
done
</pre>
<p>Verify <code>deadline</code> scheduler is in use:</p>
<pre>
root@localhost:~# cat /sys/block/sda/queue/scheduler
noop [deadline] cfq
</pre>
<p></p>
<h3>/etc/fstab options</h3>
<p>Add <code>discard</code>,<code>noatime</code>, and <code>nodiratime</code> options to all partitions on the SSD in the <code>/etc/fstab</code> file.</p>
<p>Descriptions of the options found in the <code>mount</code> man page:</p>
<pre>
discard/nodiscard
Controls whether ext4 should issue discard/TRIM commands to the underlying block device
when blocks are freed. This is useful for SSD devices and sparse/thinly-provisioned
LUNs, but it is off by default until sufficient testing has been done.
noatime
Do not update inode access times on this filesystem (e.g., for faster access on the news
spool to speed up news servers).
nodiratime
Do not update directory inode access times on this filesystem.
</pre>
<p>Backup <code>/etc/fstab</code> first:</p>
<pre>
root@localhost:~# cp -v /etc/fstab{,.bak}
‘/etc/fstab’ -> ‘/etc/fstab.bak’
</pre>
<p>Edit and add options in <code>/etc/fstab</code>:</p>
<pre>
# <file system> <mount point> <type> <options> <dump> <pass>
/dev/mapper/vg0-lv_root / ext4 errors=remount-ro,discard,noatime,nodiratime 0 1
</pre>
<p></p>
<h3>References</h3>
<p>
http://www.howtogeek.com/62761/how-to-tweak-your-ssd-in-ubuntu-for-better-performance/<br>
http://apcmag.com/how-to-maximise-ssd-performance-with-linux.htm
</p>
Rodolf Sabalburohttp://www.blogger.com/profile/06648192806453188759noreply@blogger.com1tag:blogger.com,1999:blog-8509744489347868336.post-71351049588931178032014-07-02T03:07:00.000-07:002014-07-06T06:23:27.441-07:00Configuring OpenVPN and SELinux<h3>
up and down user scripts
</h3>
<p>OpenVPN supports custom user scripts to be run as the service is started or stopped — known as the <code>--up</code> and <code>--down</code> parameters in the man page. On CentOS or other Red Hat-based distributions, these scripts may not work correctly because SELinux is preventing them from executing, which results in the OpenVPN service failing to start.</p>
<pre>
[root@localhost ~]# service openvpn start
Starting openvpn: [FAILED]
[root@localhost ~]# tail -15 /var/log/openvpn.log
Tue Jul 1 10:48:44 2014 TUN/TAP TX queue length set to 100
Tue Jul 1 10:48:44 2014 do_ifconfig, tt->ipv6=0, tt->did_ifconfig_ipv6_setup=0
Tue Jul 1 10:48:44 2014 /sbin/ip link set dev tun0 up mtu 1500
Tue Jul 1 10:48:44 2014 /sbin/ip addr add dev tun0 local 172.16.1.1 peer 172.16.1.2
Tue Jul 1 10:48:44 2014 PLUGIN_CALL: POST /usr/lib64/openvpn/plugins/openvpn-plugin-down-root.so/PLUGIN_UP status=0
Tue Jul 1 10:48:44 2014 /etc/openvpn/scripts/up.sh tun0 1500 1542 172.16.1.1 172.16.1.2 init
/etc/openvpn/scripts/up.sh: line 14: /sbin/iptables: Permission denied
/etc/openvpn/scripts/up.sh: line 15: /sbin/iptables: Permission denied
/etc/openvpn/scripts/up.sh: line 16: /sbin/iptables: Permission denied
/etc/openvpn/scripts/up.sh: line 17: /sbin/iptables: Permission denied
/etc/openvpn/scripts/up.sh: line 18: /sbin/iptables: Permission denied
/etc/openvpn/scripts/up.sh: line 19: /sbin/iptables: Permission denied
/etc/openvpn/scripts/up.sh: line 20: /proc/sys/net/ipv4/ip_forward: Permission denied
Tue Jul 1 10:48:44 2014 WARNING: Failed running command (--up/--down): external program exited with error status: 1
Tue Jul 1 10:48:44 2014 Exiting due to fatal error
</pre>
<ul>
<li>OpenVPN fails to start with the <code>service</code> command.</li>
<li>Error messages in <code>/var/log/openvpn.log</code> show failed execution of commands regarding the <code>up.sh</code> script.</li>
</ul>
<p>Also to further verify SELinux is causing this problem, error messages can be found in <code>/var/log/audit/audit.log</code>:</p>
<pre>
type=AVC msg=audit(1404247863.966:166): avc: denied { getattr } for pid=2946 comm="up.sh" path="/sbin/iptables-multi-1.4.7" dev=sda1 ino=705 scontext=unconfined_u:system_r:openvpn_t:s0 tcontext=system_u:object_r:iptables_exec_t:s0 tclass=file
type=SYSCALL msg=audit(1404247863.966:166): arch=c000003e syscall=4 success=no exit=-13 a0=26c6890 a1=7fff1691d490 a2=7fff1691d490 a3=50 items=0 ppid=2939 pid=2946 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=2 comm="up.sh" exe="/bin/bash" subj=unconfined_u:system_r:openvpn_t:s0 key=(null)
</pre>
<ul>
<li>Notice <code>up.sh</code> was trying to run shell commands and <code>/bin/bash</code>.</li>
</ul>
<p>To allow custom scripts to be executed two things must be done:</p>
<ol>
<li>The up or down script must have the correct file contexts set.</li>
<li>The correct SELinux boolean must be set to allow for unconfined scripts to be run by OpenVPN.</li>
</ol>
<h4>1. OpenVPN SELinux file contexts</h4>
<p>Anytime the <code>restorecon</code> command is run, it reads the <code>/etc/selinux/targeted/contexts/files/file_contexts</code> file to know what file context to set on a regular file or directory. Therefore we can search through this file to find all the default file contexts for OpenVPN using <code>grep -i openvpn /etc/selinux/targeted/contexts/files/file_contexts</code>, see below.
</p>
<pre>
[root@localhost ~]# grep -i openvpn /etc/selinux/targeted/contexts/files/file_contexts
/etc/openvpn(/.*)? system_u:object_r:openvpn_etc_t:s0
/var/log/openvpn.* system_u:object_r:openvpn_var_log_t:s0
/etc/openvpn/ipp.txt -- system_u:object_r:openvpn_etc_rw_t:s0
/var/lib/openvpn(/.*)? system_u:object_r:openvpn_var_lib_t:s0
/var/run/openvpn(/.*)? system_u:object_r:openvpn_var_run_t:s0
/etc/openvpn/scripts(/.*)? system_u:object_r:openvpn_unconfined_script_exec_t:s0
/usr/sbin/openvpn -- system_u:object_r:openvpn_exec_t:s0
/etc/rc\.d/init\.d/openvpn -- system_u:object_r:openvpn_initrc_exec_t:s0
/usr/share/munin/plugins/openvpn -- system_u:object_r:munin_services_plugin_exec_t:s0
</pre>
<p>Note all the different context types, though, the type we are most interested in is <code>system_u:object_r:openvpn_unconfined_script_exec_t:s0</code> which applies to all files and directories under <code>/etc/openvpn/scripts/</code>, and the directory itself.</p>
<p>Create the <code>/etc/openvpn/scripts</code> directory, move your custom script to it, and run a recursive <code>restorecon</code>:</p>
<pre>
[root@localhost ~]# mkdir /etc/openvpn/scripts
[root@localhost ~]# ls -Zd /etc/openvpn/scripts/
drwx------. root root unconfined_u:object_r:openvpn_etc_t:s0 /etc/openvpn/scripts/
[root@localhost ~]# mv -v up.sh /etc/openvpn/scripts/
`up.sh' -> `/etc/openvpn/scripts/up.sh'
[root@localhost ~]# restorecon -Rv /etc/openvpn/scripts/
restorecon reset /etc/openvpn/scripts context unconfined_u:object_r:openvpn_etc_t:s0->unconfined_u:object_r:openvpn_unconfined_script_exec_t:s0
restorecon reset /etc/openvpn/scripts/roadwarrior context unconfined_u:object_r:openvpn_etc_t:s0->unconfined_u:object_r:openvpn_unconfined_script_exec_t:s0
restorecon reset /etc/openvpn/scripts/up.sh context unconfined_u:object_r:admin_home_t:s0->unconfined_u:object_r:openvpn_unconfined_script_exec_t:s0
</pre>
<ul>
<li>Notice the file context type for the <code>scripts/</code> directory and the <code>scripts/up.sh</code> switch from <code>openvpn_etc_t</code> to <code>openvpn_unconfined_script_exec_t</code>.</li>
</ul>
<p>Verify the directory and files have the correct file contexts with <code>ls -Z</code>:</p>
<pre>
[root@localhost ~]# ls -Zd /etc/openvpn/scripts/
drwx------. root root unconfined_u:object_r:openvpn_unconfined_script_exec_t:s0 /etc/openvpn/scripts/
[root@localhost ~]# ls -Zl /etc/openvpn/scripts/
total 3
-rwxr-xr-x. 1 unconfined_u:object_r:openvpn_unconfined_script_exec_t:s0 root root 0 Jul 1 12:21 up.sh
</pre>
<p></p>
<h4>2. OpenVPN SELinux booleans</h4>
<p>Find the relevant SELinux booleans by listening them using <code>getsebool -a</code>:</p>
<pre>
[root@localhost ~]# getsebool -a | grep -i openvpn
openvpn_enable_homedirs --> on
openvpn_run_unconfined --> off
</pre>
<p>Change the <code>openvpn_run_unconfined</code> boolean to <em>on</em> with the <code>setsebool</code> command and verify:</p>
<pre>
[root@localhost ~]# setsebool openvpn_run_unconfined on
[root@localhost ~]# getsebool -a | grep -i openvpn
openvpn_enable_homedirs --> on
openvpn_run_unconfined --> on
</pre>
<p>Now the OpenVPN service should be able to start without any errors relating to the execution of <code>--up</code> or <code>--down</code> scripts:</p>
<pre>
[root@localhost ~]# service openvpn start
Starting openvpn: [ OK ]
[root@localhost ~]# tail -15 /var/log/openvpn.log
Tue Jul 1 12:33:45 2014 TUN/TAP TX queue length set to 100
Tue Jul 1 12:33:45 2014 do_ifconfig, tt->ipv6=0, tt->did_ifconfig_ipv6_setup=0
Tue Jul 1 12:33:45 2014 /sbin/ip link set dev tun0 up mtu 1500
Tue Jul 1 12:33:45 2014 /sbin/ip addr add dev tun0 local 172.16.1.1 peer 172.16.1.2
Tue Jul 1 12:33:45 2014 PLUGIN_CALL: POST /usr/lib64/openvpn/plugins/openvpn-plugin-down-root.so/PLUGIN_UP status=0
Tue Jul 1 12:33:45 2014 /etc/openvpn/scripts/up.sh tun0 1500 1542 172.16.1.1 172.16.1.2 init
Tue Jul 1 12:33:46 2014 /sbin/ip route add 172.16.1.0/24 via 172.16.1.2
Tue Jul 1 12:33:46 2014 GID set to nobody
Tue Jul 1 12:33:46 2014 UID set to nobody
Tue Jul 1 12:33:46 2014 UDPv4 link local (bound): [undef]
Tue Jul 1 12:33:46 2014 UDPv4 link remote: [undef]
Tue Jul 1 12:33:46 2014 MULTI: multi_init called, r=256 v=256
Tue Jul 1 12:33:46 2014 IFCONFIG POOL: base=172.16.1.4 size=62, ipv6=0
Tue Jul 1 12:33:46 2014 IFCONFIG POOL LIST
Tue Jul 1 12:33:46 2014 Initialization Sequence Completed
</pre>
<p></p>
<h3>
Custom OpenVPN listening port
</h3>
<p>
The default listening port for OpenVPN is UDP port 1194. If you decide to use another port, SELinux settings must be updated using <code>semanage</code> to reflect the newly assigned port.
</p>
<p>List SELinux port information and find port contexts matching OpenVPN with <code>semanage port -l | grep -i openvpn</code>:</p>
<pre>
[root@localhost ~]# semanage port -l | grep -i openvpn
openvpn_port_t tcp 1194
openvpn_port_t udp 1194
</pre>
<p>If we decide to use UDP port 2194 instead, use the following <code>semanage</code> command:</p>
<pre>
[root@localhost ~]# semanage port -a -t openvpn_port_t -p udp 2194
</pre>
<p>Optionally you can swap <em>udp</em> with <em>tcp</em> if you are using OpenVPN in TCP mode.</p>
<p>Verify SELinux port contexts are updated:</p>
<pre>
[root@localhost ~]# semanage port -l | grep -i openvpn
openvpn_port_t tcp 1194
openvpn_port_t udp 2194, 1194
</pre>
<p></p>
<h3>References</h3>
<p>
http://sysadmin-notepad.blogspot.com/2013/05/custom-openvpn-port-with-selinux-enabled.html<br>
http://www.mankier.com/8/openvpn_unconfined_script_selinux
</p>Rodolf Sabalburohttp://www.blogger.com/profile/06648192806453188759noreply@blogger.com0tag:blogger.com,1999:blog-8509744489347868336.post-51941724993583391512011-09-19T18:33:00.001-07:002014-07-04T04:20:21.689-07:00Sysadmin guide to using git and etckeeper for tracking configuration changesWhen administering any Linux or Unix system over a period time, the problem of tracking what changes occurred to which files quickly arises. This becomes increasingly difficult when there are multiple system administrators configuring a single server. How do you easily track who made what changes, and when? This is where <a href="http://git-scm.com/">git</a> comes in handy.<br /><br />Git is a distributed revision version control system, which enables you to track changes to a file(s) by using a repository, also known as an index. Changes are tracked each <i>commit</i>. A commit can be thought of as a snapshot of a file's current state, similar to virtual machine snapshots in a virtual environment.<br /><br />The index, which is basically the main database, can be initiated in any directory. However, wherever the Git index is initiated, it will <i>only</i> track files that are a child of that directory. For example, if we were to track /home/localadmin, it would track all files and sub-directories within localadmin's home directory, but nothing above it in the directory tree. A system can have many indexes initiated in different directories.<br /><br />On Linux and Unix servers, changes primarily occur within the /etc directory. This is where most of the important .conf files are found. When a new package is installed, existing files may change and several new ones are added to /etc. All of these files and changes will eventually need to be automatically committed and added to the git index. This is where etckeeper comes in.<br /><br />Etckeeper works with git, allowing automatic commits to the /etc directory after each package installation (shown and explained later.) It also tracks file metadeta that revision control systems do not normally support, but are important for /etc, such as permissions of /etc/shadow.<br /><br />You may have heard of git heavily in the software development community, but I try to show how it can also be leveraged as a powerful tool for system administrators.<br /><br /><b>NOTE</b>: This tutorial demonstrates instructions based on an Ubuntu 10.04 LTS 32-bit server installation. However, I have successfully configured the same setup on Ubuntu 11.04, Debian 5, Debian 6, CentOS 5.6, CentOS 6.0, and Red Hat Enterprise Linux 6.0.<br /><br /><font style="font-size: 160%; color: #424242; weight: bold;">Installing git and etckeeper</font><br /><br />Install the git and etckeeper packages:<br /><br /><code>$ sudo aptitude -yvVR install git-core etckeeper</code><br /><br /><ul><li>-y assume yes when prompted.</li><li>-v verbose output.</li><li>-V show which versions of packages will be installed.</li><li>-R do not treat recommended packages as dependencies.</li></ul><br />Etckeeper supports other software version control systems (VCS) besides git, double-check the configuration file to make sure it is using git.<br /><br /><code>$ sudo vi /etc/etckeeper/etckeeper.conf</code><br /><pre># The VCS to use.<br /># VCS="hg"<br />VCS="git"<br /># VCS="bzr"<br /># VCS="darcs"</pre><br />Now initialize the <code>/etc</code> directory:<br /><br /><code>$ cd /etc</code><br /><code>$ sudo etckeeper init</code><br /><br />Now make your first commit:<br /><br /><code>$ sudo git commit</code><br /><br />You will be brought to a screen where you can create a commit message along with the commit itself. Enter a brief description above the commented lines.<br /><br /><pre>Initial commit for server build<br /><br /> * Initializing git repository in /etc<br /><br /># Please enter the commit message for your changes. Lines starting<br /># with '#' will be ignored, and an empty message aborts the commit.<br />#<br /># Committer: localhost root <root@localhost.localdomain><br />#<br /># On branch master<br />#<br /># Initial commit<br />#<br /># Changes to be committed:<br /># (use "git rm --cached <file>…" to unstage)<br />#<br /># new file: .etckeeper<br /># new file: .gitignore<br /># new file: adduser.conf<br /># new file: aliases<br /># new file: alternatives/README<br /># new file: alternatives/awk<br /># new file: alternatives/awk.1.gz<br /># new file: alternatives/builtins.7.gz</pre><br />In each commit message, git will display which files will be commited to the repository.<br /><br /><code>sudo</code> is required for running the git commands because the repository was initialized in /etc by sudo; it is readable and writable only by root.<br /><br />Any git command should be run after changing your working directory to <code>/etc</code>.<br /><br /><font style="font-size: 140%; color: #424242; weight: bold;">Making commits</font><br /><br />After the first initialization, if a file is changed, git will notice the change.<br /><br /><code>$ sudo git status</code><br /><br /><ul><li>Show status of current repository, changes, etc.</li></ul><br /><pre>rsabalburo@localhost:/etc$ sudo git status<br /><br /># On branch master<br /># Changed but not updated:<br /># (use "git add <file>…" to update what will be committed)<br /># (use "git checkout -- <file>…" to discard changes in working directory)<br />#<br /># modified: issue<br />#<br />no changes added to commit (use "git add" and/or "git commit -a")</pre><br />Above I added the line "Add new message of the day." to the <code>/etc/issue</code> file.<br /><br />Using the following command, we can see what changes happened to <code>/etc/issue</code> since the previous commit:<br /><br /><code>$ sudo git diff</code><br /><br /><pre>rsabalburo@localhost:/etc$ sudo git diff<br /><br />diff --git a/issue b/issue<br />index 9029f91..2100bfd 100644<br />--- a/issue<br />+++ b/issue<br />@@ -1,2 +1,4 @@<br />+A new message of the day.<br />+<br /> Ubuntu 10.04 LTS n l</pre><br /><ul><li>The diff output is identical to the regular diff command (using -u unified output switch).</li><li>The + shows the changes that were added since the previous commit.</li></ul><br /><br />Now that we are satisfied with the change we can <code>git commit</code> it:<br /><br /><code>$ sudo git commit -a</code><br /><br /><ul><li>-a commit all changes.</li></ul><br />Again, you will be prompted to create a description along with your commit.<br /><br /><pre>Add a new message of the day for users logging in<br /><br /># Please enter the commit message for your changes. Lines starting<br /># with '#' will be ignored, and an empty message aborts the commit.<br />#<br /># Committer: localhost root <root@localhost.localdomain><br />#<br /># On branch master<br /># Changes to be committed:<br /># (use "git reset HEAD <file>…" to unstage)<br />#<br /># modified: issue<br />#</pre><br />If you made an error in your commit message or would simply like to change it, you can go back and edit it with <code>git commit --amend</code>:<br /><br /><code>$ sudo git commit --amend</code><br /><br /><ul><li>This will only allow you to change the commit message of your most recent commit.</li></ul><br /><b>NOTE</b>: You generally cannot amend any commit messages past the previous commit, doing so requires you to rewrite the entire git repository which is a very complicated process.<br /><br /><b>Important</b>: If you have more than one modified file showing up in the "Changes to be committed:" staging area, you can commit each single file rather than all at once (-a), which is highly recommended.<br /><br /><code>$ sudo git commit <file></code><br /><br /><font style="font-size: 140%; color: #424242; weight: bold;">How do I display commits I've made?</font><br /><br />To view a log of all the commits that you have made:<br /><br /><code>$ sudo git log</code><br /><br /><pre>rsabalburo@localhost:/etc$ sudo git log<br /><br />commit d23affc7b89da8966826ab0fef6891d87dbd3ee1<br />Author: localhost root <root@localhost.localdomain><br />Date: Thu Jun 17 12:37:55 2010 -1000<br /><br /> Add a new message of the day for users logging in<br /><br />commit 30636b4bdfb84cdd4db756df60dcc2c49b53842b<br />Author: localhost root <root@localhost.localdomain><br />Date: Thu Jun 17 11:43:47 2010 -1000<br /><br /> Initial commit for server build<br /><br /> * Initializing git repository in /etc</pre><br />Each commit message has a unique SHA-1 hash, commit author, date of commit, and description of commit.<br /><br />In the above git log, the author is currently localhost root. It is recommended that you change author and email address, especially if there are multiple administrators configuring the same server.<br /><br /><code>$ git config --global user.name "Rodolf Sabalburo"</code><br /><code>$ git config --global user.email "rsabalburo@localhost"</code><br /><br />Notice <code>sudo</code> is not needed for these commands. These commands will automatically create the <code>.gitconfig</code> file in the current users home directory:<br /><br /><pre>hccitc@localhost:~$ cd ~/<br />hccitc@localhost:~$ cat .gitconfig<br /><br />[core]<br /> editor = vim<br />[user]<br /> name = Rodolf Sabalburo<br /> email = rsabalburo@localhost</pre><br />From this point on, <code>Rodolf Sabalburo <rsabalburo@localhost></code> will display in the commit log for any commit made by the user rsabalburo:<br /><br /><pre>rsabalburo@localhost:/etc$ sudo git log<br /><br />commit cae791d4ac3aeb206f108815f7a5d4fc0a3bd49b<br />Author: Rodolf Sabalburo <rsabalburo@localhost><br />Date: Thu Jun 17 13:57:07 2010 -1000<br /><br /> Add hosts entry for 192.168.1.1<br /> <br /> * Showing different author and email in commit message.<br /><br />commit d23affc7b89da8966826ab0fef6891d87dbd3ee1<br />Author: localhost root <root@localhost.localdomain><br />Date: Thu Jun 17 12:37:55 2010 -1000<br /><br /> Add a new message of the day for users logging in<br /><br />commit 30636b4bdfb84cdd4db756df60dcc2c49b53842b<br />Author: localhost root <root@localhost.localdomain><br />Date: Thu Jun 17 11:43:47 2010 -1000<br /><br /> Initial commit for server build<br /><br /> * Initializing git repository in /etc</pre><br /><b>Important</b>: Ensure that other privileged users set their git user.name and user.email, otherwise the commit author will show up as <code>localhost root <root@localhost.localdomain></code>.<br /><br />The <code>git log</code> command supports several additional switches, one of the most important being the <i>-u</i> switch which displays a diff unified output for each commit:<br /><br /><pre>rsabalburo@localhost:/etc$ sudo git log<br /><br />commit cae791d4ac3aeb206f108815f7a5d4fc0a3bd49b<br />Author: Rodolf Sabalburo <rsabalburo@localhost><br />Date: Thu Jun 17 13:57:07 2010 -1000<br /><br /> Add hosts entry for 192.168.1.1<br /> <br /> * Showing different author and email in commit message.<br /><br />diff --git a/hosts b/hosts<br />index 4eaa7c5..12a111b 100644<br />--- a/hosts<br />+++ b/hosts<br />@@ -1,5 +1,6 @@<br /> 127.0.0.1 localhost<br /> 127.0.1.1 localhost.localdomain localhost<br />+192.168.1.1 localgateway<br /> <br /> # The following lines are desirable for IPv6 capable hosts<br /> ::1 localhost ip6-localhost ip6-loopback<br /><br />commit d23affc7b89da8966826ab0fef6891d87dbd3ee1<br />Author: localhost root <root@localhost.localdomain><br />Date: Thu Jun 17 12:37:55 2010 -1000<br /><br /> Add a new message of the day for users logging in<br /><br />diff --git a/issue b/issue<br />index 9029f91..2100bfd 100644<br />--- a/issue<br />+++ b/issue<br />@@ -1,2 +1,4 @@<br />+A new message of the day.<br />+<br /> Ubuntu 10.04 LTS n l</pre><br />It also supports the <i>--color</i> switch, <code>sudo git log -u --color</code> which displays commit messages in yellow, removed text/lines in red, added text/lines in green.<br /><br />To make the color behavior default use <code>git config</code>:<br /><br /><code>$ git config --global color.diff auto</code><br /><br />Additional <code>git log</code> switches:<br /><br /><code>sudo git log --pretty=oneline</code><br /><code>sudo git log --pretty=email</code><br /><code>sudo git log --decorate</code><br /><br />See <code>'man git-log'</code> for more and the <i>Additional git commands</i> near the end of the tutorial for examples.<br /><br /><font style="font-size: 140%; color: #424242; weight: bold;">How do I compare different commits?</font><br /><br />When viewing the different commits, you can compare changes with the <code>git diff</code> command:<br /><br /><code>$ sudo git diff -u <older-commit-hash> <newer-commit-hash></code><br /><br /><ul><li>You do not have to type the entire hash, the first 4-6 hash charactes will suffice.</li></ul><br /><pre>rsabalburo@localhost:/etc$ sudo git diff -u 30636b a208e373<br /><br />diff --git a/hosts b/hosts<br />index 4eaa7c5..b5fe3df 100644<br />--- a/hosts<br />+++ b/hosts<br />@@ -1,5 +1,7 @@<br /> 127.0.0.1 localhost<br /> 127.0.1.1 localhost.localdomain localhost<br />+192.168.1.1 localgateway<br />+192.168.1.2 localproxy<br /> <br /> # The following lines are desirable for IPv6 capable hosts<br /> ::1 localhost ip6-localhost ip6-loopback<br />diff --git a/issue b/issue<br />index 9029f91..2100bfd 100644<br />--- a/issue<br />+++ b/issue<br />@@ -1,2 +1,4 @@<br />+A new message of the day.<br />+<br /> Ubuntu 10.04 LTS n l</pre><br />You can compare commits using <code>git log</code> as well.<br /><br />By default when issuing git log under <code>/etc</code>, it will display <i>ALL</i> commits within the <code>/etc</code> git repository.<br /><br />You can run git log on a single file (or directory) to see all changes made to it:<br /><br /><code>$ sudo git log <file/directory></code><br /><br /><ul><li>-u unified diff output</li></ul><br /><pre>rsabalburo@localhost:/etc$ sudo git log -u hosts<br /><br />commit a208e373fc8c5ddff86e752128bada592ccd2f60<br />Author: Rodolf Sabalburo <rsabalburo@localhost><br />Date: Thu Jun 17 14:26:24 2010 -1000<br /><br /> Add new host entry for 192.168.1.2 localproxy<br /> <br /> * Make more changes to hosts to demonstrate git log on single file.<br /><br />diff --git a/hosts b/hosts<br />index 12a111b..b5fe3df 100644<br />--- a/hosts<br />+++ b/hosts<br />@@ -1,6 +1,7 @@<br /> 127.0.0.1 localhost<br /> 127.0.1.1 localhost.localdomain localhost<br /> 192.168.1.1 localgateway<br />+192.168.1.2 localproxy<br /> <br /> # The following lines are desirable for IPv6 capable hosts<br /> ::1 localhost ip6-localhost ip6-loopback<br /><br />commit cae791d4ac3aeb206f108815f7a5d4fc0a3bd49b<br />Author: Rodolf Sabalburo <rsabalburo@localhost><br />Date: Thu Jun 17 13:57:07 2010 -1000<br /><br /> Add hosts entry for 192.168.1.1<br /> <br /> * Showing different author and email in commit message.<br /><br />diff --git a/hosts b/hosts<br />index 4eaa7c5..12a111b 100644<br />--- a/hosts<br />+++ b/hosts<br />@@ -1,5 +1,6 @@<br /> 127.0.0.1 localhost<br /> 127.0.1.1 localhost.localdomain localhost<br />+192.168.1.1 localgateway<br /> <br /> # The following lines are desirable for IPv6 capable hosts<br /> ::1 localhost ip6-localhost ip6-loopback</pre><br /><font style="font-size: 140%; color: #424242; weight: bold;">Tracking new files</font><br /><br />Git will track new files added to the repository (any file under <code>/etc</code> because we initiated the repository here) and display a notification in it's status output.<br /><br /><pre>rsabalburo@localhost:/etc/shorewall$ sudo cp /usr/share/doc/shorewall/default-config/masq .<br /><br />rsabalburo@localhost:/etc/shorewall$ sudo git status<br /># On branch master<br /># Untracked files:<br /># (use "git add <file>…" to include in what will be committed)<br />#<br /># masq<br />nothing added to nothing added to commit but untracked files present (use "git add" to track)</pre><br />Add the file or directory using <code>git add</code>:<br /><br /><code>$ sudo git add <file/directory></code><br /><br /><pre>rsabalburo@localhost:/etc/shorewall$ sudo git add masq<br /><br />rsabalburo@localhost:/etc/shorewall$ sudo git status<br /><br /># On branch master<br /># Changes to be committed:<br /># (use "git reset HEAD <file>…" to unstage)<br />#<br /># new file: masq<br />#</pre><br />Git will now notify you that the masq file has changes to be commited. After committing the changes, <code>git status</code> will display that there are none.<br /><br /><pre>rsabalburo@localhost:/etc/shorewall$ sudo git commit -a<br />[master 23d7db8] Copy masq file to shorewall/<br /> 1 files changed, 11 insertions(+), 0 deletions(-)<br /> create mode 100644 shorewall/masq<br /><br />rsabalburo@localhost:/etc/shorewall$ sudo git status<br /># On branch master<br />nothing to commit (working directory clean)</pre><br /><font style="font-size: 140%; color: #424242; weight: bold;">How do I untrack files?</font><br /><br />If there is a file that you do not want committed or tracked by git use the <code>git rm --cached</code> command along with the <code>.gitignore</code> file.<br /><br /><pre>rsabalburo@localhost:/etc/shorewall$ sudo git rm --cached masq<br />rm 'shorewall/masq'<br /><br />rsabalburo@localhost:/etc/shorewall$ sudo git status<br /># On branch master<br /># Changes to be committed:<br /># (use "git reset HEAD <file>…" to unstage)<br />#<br /># deleted: masq<br />#<br /># Untracked files:<br /># (use "git add <file>…" to include in what will be committed)<br />#<br /># masq</pre><br />Notice that the masq file shows "deleted", this does not mean it has been deleted from the system only deleted from the git repository. The masq file still shows up under the <i>Untracked files</i> staging area.<br /><br /><pre>rsabalburo@localhost:/etc/shorewall$ sudo git commit -a<br /><br />[master bf06da0] Remove shorewall/masq from git repository<br /> 1 files changed, 0 insertions(+), 11 deletions(-)<br /> delete mode 100644 shorewall/masq<br /><br /><br />rsabalburo@localhost:/etc/shorewall$ sudo git log<br /><br />commit bf06da0f52c16002dbf1a134ffc97b8f30d22673<br />Author: Rodolf Sabalburo <rsabalburo@localhost><br />Date: Thu Jun 17 14:56:04 2010 -1000<br /><br /> Remove shorewall/masq from git repository<br /> <br /> * masq itself is not deleted from the file system, only from the<br /> repository.<br /> * To remove from staging area, use .gitignore file.<br /><br /><br />rsabalburo@localhost:/etc/shorewall$ sudo git status<br /><br /># On branch master<br /># Untracked files:<br /># (use "git add <file>…" to include in what will be committed)<br />#<br /># masq</pre><br />To untrack masq completely, add the file to <code>/etc/.gitignore</code>. This file is responsible for ignoring any files or directories to be tracked from git.<br /><br /><pre>hccitc@localhost:/etc$ sudo vi .gitignore<br /># Ignore masq file to demonstrate .gitignore<br />shorewall/masq<br /><br /><br />rsabalburo@localhost:/etc$ sudo git status<br /># On branch master<br /># Changed but not updated:<br /># (use "git add <file>…" to update what will be committed)<br /># (use "git checkout -- <file>…" to discard changes in working directory)<br />#<br /># modified: .gitignore<br />#</pre><br />Git also tracks the <code>.gitignore</code> file, commit the change describing which file was ignored in the commit message.<br /><br /><pre>rsabalburo@localhost:/etc$ sudo git commit -a<br />[master 13957e9] Add shorewall/masq to .gitignore<br /> 1 files changed, 6 insertions(+), 0 deletions(-)<br /><br />rsabalburo@localhost:/etc$ sudo git log<br />commit 13957e9dc87a5207a11081298f48ff37c1011b6b<br />Author: Rodolf Sabalburo <rsabalburo@localhost><br />Date: Thu Jun 17 15:01:12 2010 -1000<br /><br /> Add shorewall/masq to .gitignore<br /> <br /> * This file will no longer be tracked or show up in untracked files<br /> staging area.<br /><br />commit bf06da0f52c16002dbf1a134ffc97b8f30d22673<br />Author: Rodolf Sabalburo <rsabalburo@localhost><br />Date: Thu Jun 17 14:56:04 2010 -1000<br /><br /> Remove shorewall/masq from git repository<br /> <br /> * masq itself is not deleted from the file system, only from the<br /> repository.<br /> * To remove from staging area, use .gitignore file.</pre><br /><font style="font-size: 140%; color: #424242; weight: bold;">How do I reset to an older commit?</font><br /><br />In the instance you need to rollback to an earlier or snapshot, use <code> git reset --hard</code>.<br /><br /><code>$ sudo git reset --hard <commit hash></code><br /><br /><b>WARNING!</b>: Be very careful with this command! Not only does it abandon any uncommitted changes when rolling back to a previous commit i.e. any changes you have not committed will not be saved and wiped out, it also wipes out any changes in commits found after the one you are resetting to! ONLY perform a hard reset if you are absolutely sure you want to reset to an earlier commit.<br /><br />For example if I had made changes to <code>/etc/shorewall/params</code> and <code>/etc/shorewall/rules</code> and wanted to roll back prior those changes:<br /><br /><pre>rsabalburo@localhost:/etc/shorewall$ sudo git status<br /><br /># On branch master<br />nothing to commit (working directory clean)<br /><br />rsabalburo@localhost:/etc/shorewall$ sudo git log --pretty=oneline<br /><br />9f3a7708d600ed44913d1f7a6cacc0919825740f Add rule to allow ssh from net zone to $FW<br />d8c2337ed3127a9d091a3075857ae46b6d7284a2 Add iv-rover to trusted ssh jumpserver variable<br />8512a7dac4565bf6ec8fb459ca4faf0f08471429 Add shorewall/masq to .gitignore<br />bf06da0f52c16002dbf1a134ffc97b8f30d22673 Remove shorewall/masq from git repository</pre><br />The commit I would want to reset to would be "8512a… Add shorewall/masq to .gitignore" using:<br /><br /><code>$ sudo git reset --hard 8512a7d</code><br /><br /><pre>rsabalburo@localhost:/etc/shorewall$ sudo git reset --hard 8512a7d<br /><br />HEAD is now at 8512a7d Add shorewall/masq to .gitignore<br /><br /><br />rsabalburo@localhost:/etc/shorewall$ sudo git status<br /><br /># On branch master<br />nothing to commit (working directory clean)<br /><br /><br />rsabalburo@localhost:/etc/shorewall$ sudo git log<br /><br />commit 8512a7dac4565bf6ec8fb459ca4faf0f08471429<br />Author: Rodolf Sabalburo <rsabalburo@localhost><br />Date: Thu Jun 17 15:11:23 2010 -1000<br /><br /> Add shorewall/masq to .gitignore<br /> <br /> * This file will no longer be tracked or show up in untracked files<br /> staging area.</pre><br /><font style="font-size: 140%; color: #424242; weight: bold;">Undoing a reset</font><br /><br />In the event you do a hard reset, it is still possible to recover by finding the commit using <code>git reflog</code>. <code>reflog</code> is a history log of all actions regarding commits in git.<br /><br /><code>$ sudo git reflog</code><br /><br /><pre>rsabalburo@localhost:/etc/shorewall$ sudo git reflog<br /><br />8512a7d HEAD@{0}: 8512a7d: updating HEAD<br />9f3a770 HEAD@{1}: commit: Add rule to allow ssh from net zone to $FW<br />d8c2337 HEAD@{2}: commit: Add iv-rover to trusted ssh jumpserver variable<br />8512a7d HEAD@{3}: commit: Add shorewall/masq to .gitignore</pre><br /><ul><li>Notice "updating HEAD" in reference to commit "8512a7d Add shorewall/masq to .gitignore", which we previously reset to.</li><li>We want to roll back to 9f3a770 the most recent commit before our reset.</li></ul><br /><pre>rsabalburo@localhost:/etc/shorewall$ sudo git reset --hard 9f3a770<br /><br />HEAD is now at 9f3a770 Add rule to allow ssh from net zone to $FW<br /><br /><br />rsabalburo@localhost:/etc/shorewall$ sudo git status<br /><br /># On branch master<br />nothing to commit (working directory clean)<br /><br /><br />rsabalburo@localhost:/etc/shorewall$ sudo git log<br /><br />commit 9f3a7708d600ed44913d1f7a6cacc0919825740f<br />Author: Rodolf Sabalburo <rsabalburo@localhost><br />Date: Thu Jun 17 15:15:48 2010 -1000<br /><br /> Add rule to allow ssh from net zone to $FW<br /><br />commit d8c2337ed3127a9d091a3075857ae46b6d7284a2<br />Author: Rodolf Sabalburo <rsabalburo@localhost><br />Date: Thu Jun 17 15:15:17 2010 -1000<br /><br /> Add iv-rover to trusted ssh jumpserver variable<br /><br />commit 8512a7dac4565bf6ec8fb459ca4faf0f08471429<br />Author: Rodolf Sabalburo <rsabalburo@localhost><br />Date: Thu Jun 17 15:11:23 2010 -1000<br /><br /> Add shorewall/masq to .gitignore<br /> <br /> * This file will no longer be tracked or show up in untracked files<br /> staging area.</pre><br />The git repository head is now at the commit before our initial reset.<br /><br /><font style="font-size: 140%; color: #424242; weight: bold;">How do I reset to the previous commit?</font><br /><br />If you just want to reset or revert a just a <i>SINGLE</i> file to the previous commit use:<br /><br /><code>$ sudo git checkout <filename></code><br /><br />For example, lets say I add a PING rule to <code>/etc/shorewall/rules</code> and test it out. I did not commit the changes yet because they didn't work -- I can simply reset the single file to the most previous commit.<br /><br /><pre>rsabalburo@localhost:/etc/shorewall$ sudo git status<br /><br /># On branch master<br /># Changed but not updated:<br /># (use "git add <file>…" to update what will be committed)<br /># (use "git checkout -- <file>…" to discard changes in working directory)<br />#<br /># modified: rules<br />#<br />no changes added to commit (use "git add" and/or "git commit -a")<br /><br /><br />rsabalburo@localhost:/etc/shorewall$ sudo git diff<br /><br />diff --git a/shorewall/rules b/shorewall/rules<br />index 09dfd26..b47ea62 100644<br />--- a/shorewall/rules<br />+++ b/shorewall/rules<br />@@ -20,7 +20,8 @@ SSH(ACCEPT) net $FW<br /> <br /> # Drop Ping from the "bad" net zone.. and prevent your log from being flooded..<br /> <br />-Ping(DROP) net $FW<br />+#Ping(DROP) net $FW<br />+Ping(ACCEPT) net $FW<br /> <br /> # Permit all ICMP traffic FROM the firewall TO the net zone<br /><br /> <br />rsabalburo@localhost:/etc/shorewall$ sudo git checkout rules<br /><br />rsabalburo@localhost:/etc/shorewall$ sudo git status<br /><br /># On branch master<br />nothing to commit (working directory clean)</pre><br />If you just want to reset or revert <i>ALL</i> files (which were changed, but uncommited) to the previous commit use:<br /><br /><code>$ sudo git reset --hard</code><br /><br />In this example, lets say I add a PING rule to <code>/etc/shorewall/rules</code> and change a policy in <code>/etc/shorewall/policy</code>. If I test out these changes and I'm not satisfied, I can reset everything to the most previous commit rather than having to use <code>git checkout</code>on each individual file.<br /><br /><pre>rsabalburo@localhost:/etc/shorewall$ sudo git status<br /><br /># On branch master<br /># Changed but not updated:<br /># (use "git add <file>…" to update what will be committed)<br /># (use "git checkout -- <file>…" to discard changes in working directory)<br />#<br /># modified: policy<br /># modified: rules<br />#<br />no changes added to commit (use "git add" and/or "git commit -a")<br /><br /><br />rsabalburo@localhost:/etc/shorewall$ sudo git diff<br /><br />diff --git a/shorewall/policy b/shorewall/policy<br />index dd1d7a6..f43371a 100644<br />--- a/shorewall/policy<br />+++ b/shorewall/policy<br />@@ -15,8 +15,8 @@<br /> <br /> # RS: If you want open access to the Internet from the firewall, change the<br /> # $FW to net policy to 'ACCEPT' and remove 'info' under LOG LEVEL.<br />-#$FW net ACCEPT<br />-$FW net REJECT info<br />+$FW net ACCEPT<br />+#$FW net REJECT info<br /> net all DROP info<br /> # The FOLLOWING POLICY MUST BE LAST<br /> all all REJECT info<br />diff --git a/shorewall/rules b/shorewall/rules<br />index 09dfd26..b47ea62 100644<br />--- a/shorewall/rules<br />+++ b/shorewall/rules<br />@@ -20,7 +20,8 @@ SSH(ACCEPT) net $FW<br /> <br /> # Drop Ping from the "bad" net zone.. and prevent your log from being flooded..<br /> <br />-Ping(DROP) net $FW<br />+#Ping(DROP) net $FW<br />+Ping(ACCEPT) net $FW<br /> <br /> # Permit all ICMP traffic FROM the firewall TO the net zone<br /><br /><br /> <br />rsabalburo@localhost:/etc/shorewall$ sudo git reset --hard<br /><br />HEAD is now at 9f3a770 Add rule to allow ssh from net zone to $FW<br /><br /><br />rsabalburo@localhost:/etc/shorewall$ sudo git status<br /><br /># On branch master<br />nothing to commit (working directory clean)</pre><br /><font style="font-size: 140%; color: #424242; weight: bold;">How do I reset a SINGLE file to a much older commit?</font><br /><br />If you have a single file you want to revert to a much older commit e.g. maybe you want a revert a configuration file to it's original form in one of your first commits, use the <code>git checkout</code> command and specify the hash and filename as arguments.<br /><br /><code> sudo git checkout <hash> <filename></code><br /><br /><ul><li>Below I use <code>sudo git checkout cae7 hosts</code>, <code>cae7</code> is the <i>hash</i>, <code>hosts</code> is the <i>filename</i>.</li><li><br />I found the correct hash I wanted to revert to by using <code>sudo git log -u hosts</code>.</li></ul><br /><pre>rsabalburo@localhost:/etc$ sudo git status<br /><br /># On branch master<br />nothing to commit (working directory clean)<br /><br /><br />rsabalburo@localhost:/etc$ head hosts<br /><br />127.0.0.1 localhost<br />127.0.1.1 localhost.localdomain localhost<br />192.168.1.1 localgateway<br />192.168.1.2 localproxy<br /><br /># The following lines are desirable for IPv6 capable hosts<br />::1 localhost ip6-localhost ip6-loopback<br />fe00::0 ip6-localnet<br />ff00::0 ip6-mcastprefix<br />ff02::1 ip6-allnodes<br /><br /><br />rsabalburo@localhost:/etc$ sudo git show cae7<br /><br />commit cae791d4ac3aeb206f108815f7a5d4fc0a3bd49b<br />Author: Rodolf Sabalburo <rsabalburo@localhost><br />Date: Thu Jun 17 13:57:07 2010 -1000<br /><br /> Add hosts entry for 192.168.1.1<br /> <br /> * Showing different author and email in commit message.<br /><br />diff --git a/hosts b/hosts<br />index 4eaa7c5..12a111b 100644<br />--- a/hosts<br />+++ b/hosts<br />@@ -1,5 +1,6 @@<br /> 127.0.0.1 localhost<br /> 127.0.1.1 localhost.localdomain localhost<br />+192.168.1.1 localgateway<br /> <br /> # The following lines are desirable for IPv6 capable hosts<br /> ::1 localhost ip6-localhost ip6-loopback<br /><br /><br />rsabalburo@localhost:/etc$ sudo git checkout cae7 hosts<br /><br /><br />rsabalburo@localhost:/etc$ sudo git status<br /><br /># On branch master<br /># Changes to be committed:<br /># (use "git reset HEAD <file>…" to unstage)<br />#<br /># modified: hosts<br />#<br /><br /><br />rsabalburo@localhost:/etc$ head hosts<br /><br />127.0.0.1 localhost<br />127.0.1.1 localhost.localdomain localhost<br />192.168.1.1 localgateway<br /><br /># The following lines are desirable for IPv6 capable hosts<br />::1 localhost ip6-localhost ip6-loopback<br />fe00::0 ip6-localnet<br />ff00::0 ip6-mcastprefix<br />ff02::1 ip6-allnodes<br />ff02::2 ip6-allrouters</pre><br />Git shows in the staging area that hosts has been modified (in this case, to our previous commit cae7…). The hosts file would still need to be commited if we were satisfied with the changes. If not satisfied, we can reset to the previous commit using <code>git reset --hard</code>.<br /><br /><font style="font-size: 140%; color: #424242; weight: bold;">How do I modify my previous commit and include additional changes without using a reset?</font><br /><br />For example if I added the '192.168.1.3 homecomputer' host entry to <code>/etc/hosts</code> and commit the changes:<br /><br /><pre>rsabalburo@localhost:/etc$ head /etc/hosts<br /><br />127.0.0.1 localhost<br />127.0.1.1 localhost.localdomain localhost<br />192.168.1.1 localgateway<br />192.168.1.2 localproxy<br />192.168.1.3 homecomputer<br /><br /># The following lines are desirable for IPv6 capable hosts<br />::1 localhost ip6-localhost ip6-loopback<br />fe00::0 ip6-localnet<br />ff00::0 ip6-mcastprefix<br /><br /><br />rsabalburo@localhost:/etc$ sudo git log | head<br /><br />commit c853bfcc0fd65e37be4959f208af8122e1390806<br />Author: Rodolf Sabalburo <rsabalburo@localhost><br />Date: Thu Jun 24 12:08:04 2010 -1000<br /><br /> Add new host entry for 192.168.1.3 homecomputer<br /><br />commit ac1d59be8222ff8f0d956434c2c2ba57e938047e<br />Author: Rodolf Sabalburo <rsabalburo@localhost><br />Date: Wed Jun 23 13:55:15 2010 -1000<br /><br /> committing changes in /etc after apt run<br /> <br /> Package changes:<br /> +liblzo2-2 2.03-2<br /> +libpkcs11-helper1 1.07-1build1<br /> +openssl-blacklist 0.5-2<br /> +openvpn 2.1.0-1ubuntu1<br /> +openvpn-blacklist 0.4</pre><br />If I felt that I could have included another host entry to the same commit I could use <code>git commit --amend</code> on the file after I made the changes:<br /><br /><code>$ sudo git commit --amend <filename></code><br /><br /><pre>rsabalburo@localhost:/etc$ sudo vi hosts<br /><br />(Here I added the homeserver entry)<br /><br />rsabalburo@localhost:/etc$ sudo git diff<br /><br />diff --git a/hosts b/hosts<br />index 77fdaee..0f0ac1f 100644<br />--- a/hosts<br />+++ b/hosts<br />@@ -3,6 +3,7 @@<br /> 192.168.1.1 localgateway<br /> 192.168.1.2 localproxy<br /> 192.168.1.3 homecomputer<br />+192.168.1.4 homeserver<br /> <br /> # The following lines are desirable for IPv6 capable hosts<br /> ::1 localhost ip6-localhost ip6-loopback<br /><br /><br />rsabalburo@localhost:/etc$ sudo git status<br /><br /># On branch master<br /># Changed but not updated:<br /># (use "git add <file>…" to update what will be committed)<br /># (use "git checkout -- <file>…" to discard changes in working directory)<br />#<br /># modified: hosts<br />#<br />no changes added to commit (use "git add" and/or "git commit -a")<br /><br /><br />rsabalburo@localhost:/etc$ sudo git log<br /><br />commit c853bfcc0fd65e37be4959f208af8122e1390806<br />Author: Rodolf Sabalburo <rsabalburo@localhost><br />Date: Thu Jun 24 12:08:04 2010 -1000<br /><br /> Add new host entry for 192.168.1.3 homecomputer<br /><br />commit ac1d59be8222ff8f0d956434c2c2ba57e938047e<br />Author: Rodolf Sabalburo <rsabalburo@localhost><br />Date: Wed Jun 23 13:55:15 2010 -1000<br /><br /> committing changes in /etc after apt run<br /> <br /> Package changes:<br /> +liblzo2-2 2.03-2<br /> +libpkcs11-helper1 1.07-1build1<br /> +openssl-blacklist 0.5-2<br /> +openvpn 2.1.0-1ubuntu1<br /> +openvpn-blacklist 0.4<br /><br /><br />rsabalburo@localhost:/etc$ sudo git commit --amend /etc/hosts<br /><br />(update our new change, create a commit message)<br /><br />[master bfdb9ff] Add new host entry for 192.168.1.3 homecomputer and 192.168.1.4 homeserver<br /> 2 files changed, 2 insertions(+), 2 deletions(-)<br /><br /><br />rsabalburo@localhost:/etc$ sudo git log<br /><br />commit bfdb9ffc7956a6123d447f0996b9f23b6858abce<br />Author: Rodolf Sabalburo <rsabalburo@localhost><br />Date: Thu Jun 24 12:08:04 2010 -1000<br /><br /> Add new host entry for 192.168.1.3 homecomputer and 192.168.1.4 homeserver<br /><br />commit ac1d59be8222ff8f0d956434c2c2ba57e938047e<br />Author: Rodolf Sabalburo <rsabalburo@localhost><br />Date: Wed Jun 23 13:55:15 2010 -1000<br /><br /> committing changes in /etc after apt run<br /> <br /> Package changes:<br /> +liblzo2-2 2.03-2<br /> +libpkcs11-helper1 1.07-1build1<br /> +openssl-blacklist 0.5-2<br /> +openvpn 2.1.0-1ubuntu1<br /> +openvpn-blacklist 0.4</pre><br />Notice the previous commit was overwritten after we used the <code>git commit --amend</code> command.<br /><br />You can also include changes made to several other files if needed:<br /><br /><code>$ sudo git commit --amend <file1> <file2></code><br /><br /><ul><li>Redo previous commit, and include changes made to <file1>, <file2>, etc.</li></ul><br /><font style="font-size: 140%; color: #424242; weight: bold;">etckeeper</font><br /><br />Etckeeper performs the simple role of automatically committing changes within <code>/etc</code> after package installations, shown below:<br /><br /><pre>rsabalburo@localhost:/etc$ sudo git status<br /><br /># On branch master<br />nothing to commit (working directory clean)<br /><br /><br />rsabalburo@localhost:/etc$ sudo aptitude -yvVR install openvpn<br /><br />Reading package lists… Done<br />Building dependency tree <br />Reading state information… Done<br />Reading extended state information <br />Initializing package states… Done<br />Writing extended state information… Done<br />The following NEW packages will be installed:<br /> liblzo2-2{a} [2.03-2] libpkcs11-helper1{a} [1.07-1build1] openssl-blacklist{a} [0.5-2] openvpn [2.1.0-1ubuntu1] openvpn-blacklist{a} [0.4] <br />The following packages are SUGGESTED but will NOT be installed:<br /> resolvconf <br />0 packages upgraded, 5 newly installed, 0 to remove and 25 not upgraded.<br />Need to get 0B/7,937kB of archives. After unpacking 16.3MB will be used.<br />Writing extended state information… Done<br />Preconfiguring packages …<br />(Reading database … 44756 files and directories currently installed.)<br />Unpacking openssl-blacklist (from …/openssl-blacklist_0.5-2_all.deb) …<br />Unpacking liblzo2-2 (from …/liblzo2-2_2.03-2_i386.deb) …<br />Unpacking libpkcs11-helper1 (from …/libpkcs11-helper1_1.07-1build1_i386.deb) …<br />Unpacking openvpn-blacklist (from …/openvpn-blacklist_0.4_all.deb) …<br />Unpacking openvpn (from …/openvpn_2.1.0-1ubuntu1_i386.deb) …<br />Processing triggers for man-db …<br />Processing triggers for ureadahead …<br />Setting up openssl-blacklist (0.5-2) …<br />Setting up liblzo2-2 (2.03-2) …<br /><br />Setting up libpkcs11-helper1 (1.07-1build1) …<br /><br />Setting up openvpn-blacklist (0.4) …<br />Setting up openvpn (2.1.0-1ubuntu1) …<br /> * Restarting virtual private network daemon(s)… * No VPN is running.<br /><br />Processing triggers for libc-bin …<br />ldconfig deferred processing now taking place<br />[master ac1d59b] committing changes in /etc after apt run<br /> Author: Rodolf Sabalburo <rsabalburo@localhost><br /> 13 files changed, 412 insertions(+), 0 deletions(-)<br /> create mode 100644 bash_completion.d/openvpn<br /> create mode 100644 default/openvpn<br /> create mode 100755 init.d/openvpn<br /> create mode 100755 network/if-down.d/openvpn<br /> create mode 100755 network/if-up.d/openvpn<br /> create mode 100755 openvpn/update-resolv-conf<br /> create mode 120000 rc0.d/K80openvpn<br /> create mode 120000 rc1.d/K80openvpn<br /> create mode 120000 rc2.d/S16openvpn<br /> create mode 120000 rc3.d/S16openvpn<br /> create mode 120000 rc4.d/S16openvpn<br /> create mode 120000 rc5.d/S16openvpn<br /> create mode 120000 rc6.d/K80openvpn<br />Reading package lists… Done <br />Building dependency tree <br />Reading state information… Done<br />Reading extended state information <br />Initializing package states… Done<br />Writing extended state information… Done<br /><br />Current status: 0 broken [+0], 25 updates [+0], 29045 new [+0].<br /><br /><br />rsabalburo@localhost:/etc$ sudo git show<br /><br />commit ac1d59be8222ff8f0d956434c2c2ba57e938047e<br />Author: Rodolf Sabalburo <rsabalburo@localhost><br />Date: Wed Jun 23 13:55:15 2010 -1000<br /><br /> committing changes in /etc after apt run<br /> <br /> Package changes:<br /> +liblzo2-2 2.03-2<br /> +libpkcs11-helper1 1.07-1build1<br /> +openssl-blacklist 0.5-2<br /> +openvpn 2.1.0-1ubuntu1<br /> +openvpn-blacklist 0.4</pre><br /><ul><li>Notice the "create mode"s on files that were added to the system.</li><li>Also an automatic commit message is generated show in the git log.</li><li>If you want to change this message and be more verbose on the packages installed you can use <code>sudo git commit --amend</code>.</li></ul><br /><font style="font-size: 160%; color: #424242; weight: bold;">Stanardized git commit message format</font><br /><br />Here I have standardized my git commit messages to keep git logs uniform, tidy, and easy to read.<br /><br /><ul><li>The first line should be a summary or short phrase of what change was made on which file.</li><li>Use present tense only, and omit period or other punctuation at end.</li><li>Reference the file from relative path, (everything is a child of /etc) e.g. shorewall/rules.</li><li>Second line should be blank.</li><li>Third line indented with three spaces, use an asterisk * as a bullet to describe in more detail about change.</li><li>Feel free to be as verbose as possible, and make notes for self and other administrators.</li><li>Use proper punctuation and grammar.</li></ul><br />Example of a good commit message:<br /><br /><pre>Add SNMP rule from net zone to firewall zone in shorewall/rules<br /><br /> * Allow snmp traffic to reach firewall. <br /> * This will remain commented out for now as a hook, enable when needed.<br /> * Note that Shorewall 4.0 to 4.4 have different "macro" formats.</pre><br />Example of my git log from an OpenVPN server I manage:<br /><br /><pre>commit 94591e98136adcbe10212e69b30964326044dad9<br />Author: Rodolf Sabalburo <rsabalburo@localhost><br />Date: Tue Jun 22 05:49:41 2010 -1000<br /><br /> Update shorewall/params add OpenVPN variables<br /> <br /> * Add variable for NET_OPENVPN_TRUST and ROAD_OPENVPN_TRUST.<br /> * See shorewall/params for description of variables.<br /><br />commit b2bb776167537cb6cfcead81e1cca90766e5ae80<br />Author: Rodolf Sabalburo <rsabalburo@localhost><br />Date: Tue Jun 22 05:39:23 2010 -1000<br /><br /> Fix shorewall/policy<br /> <br /> * Add explicit policy for net -> $FW drop, with logging.<br /><br />commit b28940adfdfa4c2b5370831c990d84d1364c52c1<br />Author: Rodolf Sabalburo <rsabalburo@localhost><br />Date: Tue Jun 22 05:37:08 2010 -1000<br /><br /> Fix shorewall/interfaces<br /> <br /> * Change eth0 interface to eth+.<br /> - + denotes wildcard, or "all eth interfaces".<br /><br />commit 974cd9f34c4f06ecabea414006453d610e2af0e3<br />Author: Rodolf Sabalburo <rsabalburo@localhost><br />Date: Tue Jun 22 05:27:16 2010 -1000<br /><br /> Deprecate shorewall/tunnels file<br /> <br /> * tunnels to be deprecated, use rules-hcc.inc instead to allow related<br /> inbound and outbound OpenVPN traffic.<br /> * tunnels creates open-type policies in iptables ($FW -> net udp openvpn,<br /> net -> $FW udp openvpn) that take higher priority in the net2fw chain.<br /> * rules-hcc.inc will be used to accomplish close-type policies, only<br /> allowing trusted hosts or networks from the net zone.<br /> * See "Eliminating the /etc/shorewall/tunnels file"<br /> http://www.shorewall.net/VPNBasics.html<br /><br />commit 6cbab1296f351e069857a035df077fa55911a745<br />Author: Rodolf Sabalburo <rsabalburo@localhost><br />Date: Tue Jun 22 05:19:07 2010 -1000<br /><br /> Build is now a release candidate: ubuntu1004-x86-hccvpn-rc1<br /> <br /> * Hostname change using /usr/local/sbin/hcc-ubuntu-hostname-change.</pre><br /><font style="font-size: 160%; color: #424242; weight: bold;">Additional git commands</font><br /><br /><font style="font-size: 140%; color: #424242; weight: bold;">Change git commit editor</font><br /><br />For some distributions the default editor for commit messages is nano or emacs, this can be changed with the following command to vim (or vi):<br /><br /><code>$ git config --global core.editor "vim"</code><br /><br /><ul><li>On a per user basis like user.name and user.email options; sudo not needed.</li></ul><br /><font style="font-size: 140%; color: #424242; weight: bold;">Commit verbosely</font><br /><br />You can commit verbosely i.e. include the diff of the contents being committed in the commit message screen with:<br /><br /><code>$ sudo git commit -a -v</code><br /><code>$ sudo git commit -av</code><br /><br /><ul><li>You can commit a single file if instead, replace -a with filename.</il></ul><br /><font style="font-size: 140%; color: #424242; weight: bold;">git blame</font><br /><br /><code>git blame</code> will show what revision and author last modified each line of a file. Example:<br /><br /><code>$ sudo git blame <filename></code><br /><br /><pre>rsabalburo@localhost:/etc$ sudo git blame /etc/hosts<br /><br />^30636b4 (localhost root 2010-06-17 11:43:47 -1000 1) 127.0.0.1 localhost<br />^30636b4 (localhost root 2010-06-17 11:43:47 -1000 2) 127.0.1.1 localhost.localdomain localhost<br />cae791d4 (Rodolf Sabalburo 2010-06-17 13:57:07 -1000 3) 192.168.1.1 localgateway<br />a208e373 (Rodolf Sabalburo 2010-06-17 14:26:24 -1000 4) 192.168.1.2 localproxy<br />^30636b4 (localhost root 2010-06-17 11:43:47 -1000 5) <br />^30636b4 (localhost root 2010-06-17 11:43:47 -1000 6) # The following lines are desirable for IPv6 capable hosts<br />^30636b4 (localhost root 2010-06-17 11:43:47 -1000 7) ::1 localhost ip6-localhost ip6-loopback<br />^30636b4 (localhost root 2010-06-17 11:43:47 -1000 8) fe00::0 ip6-localnet<br />^30636b4 (localhost root 2010-06-17 11:43:47 -1000 9) ff00::0 ip6-mcastprefix<br />^30636b4 (localhost root 2010-06-17 11:43:47 -1000 10) ff02::1 ip6-allnodes<br />^30636b4 (localhost root 2010-06-17 11:43:47 -1000 11) ff02::2 ip6-allrouters</pre><br /><ul><li>Notice lines <code>3</code> and <code>4</code> of <code>/etc/hosts</code> was edited by Rodolf Sabalburo, both on June 17.</li><li>The far left hand column shows the truncated commit hash.</li></ul><br /><font style="font-size: 140%; color: #424242; weight: bold;">More git log switches</font><br /><br /><code>$ sudo git log --pretty=oneline</code><br /><br /><pre>ac1d59be8222ff8f0d956434c2c2ba57e938047e committing changes in /etc after apt run<br />9f3a7708d600ed44913d1f7a6cacc0919825740f Add rule to allow ssh from net zone to $FW<br />d8c2337ed3127a9d091a3075857ae46b6d7284a2 Add iv-rover to trusted ssh jumpserver variable<br />8512a7dac4565bf6ec8fb459ca4faf0f08471429 Add shorewall/masq to .gitignore<br />bf06da0f52c16002dbf1a134ffc97b8f30d22673 Remove shorewall/masq from git repository<br />23d7db8e818ff27c789b229f47d8614a99f6ce11 Copy masq file to shorewall/<br />a208e373fc8c5ddff86e752128bada592ccd2f60 Add new host entry for 192.168.1.2 localproxy<br />cae791d4ac3aeb206f108815f7a5d4fc0a3bd49b Add hosts entry for 192.168.1.1<br />d23affc7b89da8966826ab0fef6891d87dbd3ee1 Add a new message of the day for users logging in<br />30636b4bdfb84cdd4db756df60dcc2c49b53842b Initial commit for server build.</pre><br /><code>$ sudo git log --pretty=oneline --abbrev-commit</code><br /><br /><pre>ac1d59b committing changes in /etc after apt run<br />9f3a770 Add rule to allow ssh from net zone to $FW<br />d8c2337 Add iv-rover to trusted ssh jumpserver variable<br />8512a7d Add shorewall/masq to .gitignore<br />bf06da0 Remove shorewall/masq from git repository<br />23d7db8 Copy masq file to shorewall/<br />a208e37 Add new host entry for 192.168.1.2 localproxy<br />cae791d Add hosts entry for 192.168.1.1<br />d23affc Add a new message of the day for users logging in<br />30636b4 Initial commit for server build.</pre><br /><code>$ sudo git log --pretty=email</code><br /><br /><pre>From 9f3a7708d600ed44913d1f7a6cacc0919825740f Mon Sep 17 00:00:00 2001<br />From: Rodolf Sabalburo <rsabalburo@localhost><br />Date: Thu, 17 Jun 2010 15:15:48 -1000<br />Subject: [PATCH] Add rule to allow ssh from net zone to $FW<br /><br /><br />From d8c2337ed3127a9d091a3075857ae46b6d7284a2 Mon Sep 17 00:00:00 2001<br />From: Rodolf Sabalburo <rsabalburo@localhost><br />Date: Thu, 17 Jun 2010 15:15:17 -1000<br />Subject: [PATCH] Add iv-rover to trusted ssh jumpserver variable<br /><br /><br />From 8512a7dac4565bf6ec8fb459ca4faf0f08471429 Mon Sep 17 00:00:00 2001<br />From: Rodolf Sabalburo <rsabalburo@localhost><br />Date: Thu, 17 Jun 2010 15:11:23 -1000<br />Subject: [PATCH] Add shorewall/masq to .gitignore<br /><br /> * This file will no longer be tracked or show up in untracked files<br /> staging area.<br /><br />From bf06da0f52c16002dbf1a134ffc97b8f30d22673 Mon Sep 17 00:00:00 2001<br />From: Rodolf Sabalburo <rsabalburo@localhost><br />Date: Thu, 17 Jun 2010 14:56:04 -1000<br />Subject: [PATCH] Remove shorewall/masq from git repository<br /><br /> * masq itself is not deleted from the file system, only from the<br /> repository.<br /> * To remove from staging area, use .gitignore file.</pre><br /><ul><li>See <code>man git-log</code> for additional switches.</li></ul><br /><font style="font-size: 140%; color: #424242; weight: bold;">Best practices and tips</font><br /><br /><ul><li>Git commits should be atomic i.e. make a commit anytime you make a single change to a file or similar changes across a few files.</li><li>For example, if I had the same IP I had to change in multiple configuration files I can commit it all as one with a short description describing why the IP was changed.</li><li>If I changed /etc/fstab and /etc/hosts; they should be commited separately as they are entirely unrelated.</li><li>Always set the git author immediately using git config user.name and user.email.</li><li>Use <code>git status</code> and <code>git diff</code> often to see the progress of your work, and if there were any changes you did not commit yet.</li><li>Anytime you make a change, whether a commit, revert, reset; git will always update the timestamp on the related file!</li></ul><br /><font style="font-size: 140%; color: #424242; weight: bold;">Reading diff output</font><br /><br />The <code>git log</code> command supports the diff <code>-u</code> (unified format) output, which is much easier to read, and allows the log to show differences made to each commit:<br /><br /><pre>rsabalburo@localhost:/etc/shorewall$ sudo git log -u<br /><br />commit 9f3a7708d600ed44913d1f7a6cacc0919825740f<br />Author: Rodolf Sabalburo <rsabalburo@localhost><br />Date: Thu Jun 17 15:15:48 2010 -1000<br /><br /> Add rule to allow ssh from net zone to $FW<br /><br />diff --git a/shorewall/rules b/shorewall/rules<br />index 0e33b66..09dfd26 100644<br />--- a/shorewall/rules<br />+++ b/shorewall/rules<br />@@ -14,6 +14,10 @@<br /> #ACTION SOURCE DEST PROTO DEST SOURCE ORIGINAL RATE USER/ MARK<br /> # PORT PORT(S) DEST LIMIT GROUP<br /> <br />+# Allow ssh connections to $FW<br />+<br />+SSH(ACCEPT) net $FW<br />+<br /> # Drop Ping from the "bad" net zone.. and prevent your log from being flooded..<br /> <br /> Ping(DROP) net $FW</pre><br />Notice the <code>--- a/shorewall/rules</code> and the <code>+++ b/shorewall/rules</code>. These are the files that are being compared. In this case, the older rules file and the newer rules file.<br /><br /><ul><li>Spaces are where the two files are the same. (no <b>-</b> or <b>+</b> )</li><li><b>-</b> denotes changes in the previous file, but were usually removed in the newer file.</li><li><b>+</b> denotes changes in newer file, usually additions.</li></ul><br />Also notice the format <code>@@ -14,6 +14,10 @@</code>. These are called <i>change hunks</i> or <i>chunks</i>. The <code>-14</code> denotes the first/older file line number, and the <code>6</code> is the line range. This means that at line <code>14</code> in the older file there were changes within <code>6</code> lines. The second part <code>+14</code> denotes the newer file, and <code>10</code> is again the line range. In other words, <i>line <code>14</code> onward for <code>10</code> lines there is a change</i>.<br /><br />By default the <code>-u</code> switch gives 3 lines of context, or 3 lines BEFORE and AFTER there was a change. This can be changed with the <code>-UN</code> format, where <code>N</code> is the number of lines before or after the changes that show. If you only want to see changes without surrounding context use:<br /><br /><code>$ sudo git log -U0</code><br /><br /><font style="font-size: 140%; color: #424242; weight: bold;">Resources</font><br /><br />Very helpful Git command cheat sheet: <a href="http://cheat.errtheblog.com/s/git">http://cheat.errtheblog.com/s/git</a><br />Git on Wikipedia: <a href="http://en.wikipedia.org/wiki/Git_(software)">http://en.wikipedia.org/wiki/Git_(software)</a><br />Git for the lazy: <a href="http://www.spheredev.org/wiki/Git_for_the_lazy">http://www.spheredev.org/wiki/Git_for_the_lazy</a><br />Starting git using just 10 commands: <a href="http://blog.xkoder.com/2008/08/13/git-tutorial-starting-with-git-using-just-10-commands/">http://blog.xkoder.com/2008/08/13/git-tutorial-starting-with-git-using-just-10-commands/</a><br /><br /><b>If this is your first time reading this tutorial, I recommend reading it again and practicing it hands-on to fully understand it.</b>Rodolf Sabalburohttp://www.blogger.com/profile/06648192806453188759noreply@blogger.com4tag:blogger.com,1999:blog-8509744489347868336.post-22182667758305156742011-09-15T21:10:00.000-07:002014-07-04T04:19:58.319-07:00Snort 2.9.1 on CentOS 6.0This tutorial demonstrates a <a href="http://www.snort.org">Snort</a> 2.9.1 installation on CentOS 6.0 32-bit using <a href="http://www.emergingthreats.net">Emerging Threats</a> community rules. These rules will be automatically configured and updated by <a href="http://oinkmaster.sourceforge.net">Oinkmaster</a>. <br /><br />It is important to note that Snort will be installed from RPMs, and not from source. There are also many tutorials on the web that cover a Snort installation from source in addition to MySQL, Apache2, and <a href="http://sourceforge.net/projects/barnyard/">Barnyard</a> to support the <a href="http://base.secureideas.net/screens.php">BASE</a> front-end. This can be a very daunting and discouraging task to first-time Snort users. This tutorial, however, is not another one of those. <br /><br />I try to reduce the overall complexity of learning Snort by showing a very simple setup to help those first-time Snort users get "their feet wet". <br /><br /><font style="font-size: 140%; color #424242; weight: bold;">Bridging interfaces</font><br /><br />In this setup Snort will be placed inline between a cable modem and home router.<br /><br />To use Snort inline, two separate interfaces (eth0 and eth1) will need to be bridged as a single interface (br0). Using a bridge interface will allow Snort to transparently inspect all traffic passing in and out of the network.<br /><br />Configure <code>/etc/sysconfig/network-scripts/ifcfg-eth0</code>:<br /><br /><pre>DEVICE=eth0<br />BRIDGE=br0<br />NM_CONTROLLED=no<br />USERCTL=no<br />ONBOOT=yes<br />BOOTPROTO=none</pre><br />Configure <code>/etc/sysconfig/network-scripts/ifcfg-eth1</code>:<br /><br /><pre>DEVICE=eth1<br />BRIDGE=br0<br />NM_CONTROLLED=no<br />USERCTL=no<br />ONBOOT=yes<br />BOOTPROTO=none</pre><br />Now create <code>/etc/sysconfig/network-scripts/ifcfg-br0</code>:<br /><br /><pre>DEVICE=br0<br />TYPE=Bridge<br />NM_CONTROLLED=no<br />USERCTL=no<br />ONBOOT=yes<br />BOOTPROTO=none<br />DELAY=0</pre><br />Restart the networking service:<br /><br /><code># service networking restart</code><br /><br /><font style="font-size: 140%; color: #424242; weight: bold;">Installing Snort</font><br /><br />Download the snort-2.9.1-1.F13.i386.rpm RPM from the Vincent Cojot unofficial Snort repository: <br /><br /><code># wget http://vscojot.free.fr/dist/snort/snort-2.9.1/RHEL6/i386/snort-2.9.1-14.el6.i686.rpm</code><br /><br /><ul><li>Although considered "third party" RPMs, installation is a much smoother process with them.</li><li>See <a href="http://blog.snort.org/2011/01/rpms-for-rhel5-are-available-from.html">http://blog.snort.org/2011/01/rpms-for-rhel5-are-available-from.html</a> for more information.</li></ul><br />Download and install the following dependencies needed by Snort 2.9.1 from the Vincent Cojot repository:<br /><ul><li>libdnet</li><li>libpcap library version 1.1.1</li><li>Data Acquisiion Library</li></ul><br /><code># wget http://vscojot.free.fr/dist/snort/snort-2.9.1/RHEL6/i386/libdnet-1.12-7.el6.i686.rpm</code><br /><code># wget http://vscojot.free.fr/dist/snort/snort-2.9.1/RHEL6/i386/libpcap1-1.1.1-10.el6.i686.rpm</code><br /><code># wget http://vscojot.free.fr/dist/snort/snort-2.9.1/RHEL6/i386/daq-0.6.1-10.el6.i686.rpm</code><br /><code># rpm -Uvh libdnet-1.12-7.el6.i686.rpm libpcap1-1.1.1-10.el6.i686.rpm daq-0.6.1-10.el6.i686.rpm</code><br /><br />After all dependency packages have been installed, install the Snort RPM:<br /><br /><code># rpm -Uvh snort-2.9.1-14.el6.i686.rpm</code><br /><br /><ul><li>This will automatically create the Snort user and group, startup scripts, and setup the correct directories and paths.</li></ul><br />Edit <code>/etc/snort/snort.conf</code>:<br /><br /><ul><li>Configure the HOME_NET variable with your public IP address. If using Snort behind a router, configure HOME_NET with your internal network address, e.g. 192.168.1.0/24.</li><li>Define the RULE_PATH. The SO_RULE_PATH and PREPROC_RULE_PATH can be safely commented out as they are not used by the Emerging Threat rulesets.</li><li>Configure the unified2 output, change the filename from merged.log to snort.log and remove the additional options.</li></ul><br /><pre># Setup the network addresses you are protecting<br />#ipvar HOME_NET any<br />ipvar HOME_NET 1.2.3.4/32<br /><br /># Path to your rules files (this can be a relative path)<br /># Note for Windows users: You are advised to make this an absolute path,<br /># such as: c:\snort\rules<br />#var RULE_PATH ../rules<br />#var SO_RULE_PATH ../so_rules<br />#var PREPROC_RULE_PATH ../preproc_rules<br />var RULE_PATH /etc/snort/rules<br /><br /># output unified2: filename merged.log, limit 128, nostamp, mpls_event_types, vlan_event_types<br />output unified2: filename snort.log, limit 128</pre><br />Verify that Snort will start with the following command:<br /><br /><code># snort -c /etc/snort/snort.conf -i br0</code><br /><br /><ul><li>-c denotes the configuration file.</li><li>-i denotes the interface to listen on.</li></ul><br />Although no rules have been loaded yet, Snort should display a similar message if everything is configured correctly so far:<br /><br /><pre> --== Initialization Complete ==--<br /><br /> ,,_ -*> Snort! <*-<br /> o" )~ Version 2.9.1 IPv6 GRE (Build 71) <br /> '''' By Martin Roesch & The Snort Team: http://www.snort.org/snort/snort-team<br /> Copyright (C) 1998-2011 Sourcefire, Inc., et al.<br /> Using libpcap version 1.1.1<br /> Using PCRE version: 7.8 2008-09-05<br /> Using ZLIB version: 1.2.3<br /><br /> Rules Engine: SF_SNORT_DETECTION_ENGINE Version 1.15 <Build 18><br /> Preprocessor Object: SF_SIP (IPV6) Version 1.1 <Build 1><br /> Preprocessor Object: SF_FTPTELNET (IPV6) Version 1.2 <Build 13><br /> Preprocessor Object: SF_DCERPC2 (IPV6) Version 1.0 <Build 3><br /> Preprocessor Object: SF_DNS (IPV6) Version 1.1 <Build 4><br /> Preprocessor Object: SF_SDF (IPV6) Version 1.1 <Build 1><br /> Preprocessor Object: SF_SMTP (IPV6) Version 1.1 <Build 9><br /> Preprocessor Object: SF_REPUTATION (IPV6) Version 1.1 <Build 1><br /> Preprocessor Object: SF_IMAP (IPV6) Version 1.0 <Build 1><br /> Preprocessor Object: SF_SSH (IPV6) Version 1.1 <Build 3><br /> Preprocessor Object: SF_POP (IPV6) Version 1.0 <Build 1><br /> Preprocessor Object: SF_SSLPP (IPV6) Version 1.1 <Build 4><br />Commencing packet processing (pid=10957)</pre><br /><font style="font-size: 140%; color: #424242; weight: bold;">Oinkmaster and Emerging Threats rules</font><br /><br /><a href="http://www.emergingthreats.net">Emerging Threats</a> is a fantastic community-driven project that provides free Snort rules for both personal and commercial use. They update their rulesets on a daily basis and even provide a commercial subscription with support, known as <a href="http://www.emergingthreatspro.com">Emerging Threats Pro</a>. See also <a href="http://www.emergingthreatspro.com/index.php/the-rules/ruleset-coverage.html">ruleset comparison</a>.<br /><br />Oinkmaster is a simple perl script with accompanying .conf file that allows for that automatic configuration and management of Snort rules.<br /><br />In a real-production network, not all Snort rules will be used, and certain rules may be disabled. When manually downloading and updating the newest rules, rules that have been disabled may be re-enabled again during the manual update process. Oinkmaster simplifies management of rules by allowing an administrator to keep track of which rules are disabled or enabled by default during an update as well as any other additional files to ignore.<br /><br />Download the Oinkmaster tar archive from <a href="http://oinkmaster.sourceforge.net/download.shtml">http://oinkmaster.sourceforge.net/download.shtml</a> and extract:<br /><br /><code># gunzip oinkmaster-2.0.tar.gz</code><br /><code># tar xvf oinkmaster-2.0.tar</code><br /><br />The following files will be extracted to the current working directory:<br /><br /><pre>oinkmaster-2.0<br />oinkmaster-2.0/README.templates<br />oinkmaster-2.0/oinkmaster.pl<br />oinkmaster-2.0/FAQ<br />oinkmaster-2.0/UPGRADING<br />oinkmaster-2.0/README<br />oinkmaster-2.0/README.win32<br />oinkmaster-2.0/README.gui<br />oinkmaster-2.0/LICENSE<br />oinkmaster-2.0/INSTALL<br />oinkmaster-2.0/ChangeLog<br />oinkmaster-2.0/oinkmaster.1<br />oinkmaster-2.0/template-examples.conf<br />oinkmaster-2.0/oinkmaster.conf<br />oinkmaster-2.0/contrib<br />oinkmaster-2.0/contrib/create-sidmap.pl<br />oinkmaster-2.0/contrib/addsid.pl<br />oinkmaster-2.0/contrib/makesidex.pl<br />oinkmaster-2.0/contrib/addmsg.pl<br />oinkmaster-2.0/contrib/README.contrib<br />oinkmaster-2.0/contrib/oinkgui.pl</pre><br />Change into the <code>oinkmaster-2.0/</code> directory and edit the <code>oinkmaster.conf</code> file.<br /><br />Edit <code>oinkmaster-2.0/oinkmaster.conf</code>:<br /><br /><ul><li>Add the <code>url</code> to the Emerging Threat rules Snort 2.9.0 rules. (2.9.0 rules are compatible with 2.9.1)</li><li>Set the <code>tmpdir</code> to <code>/tmp</code>.</ul><br /><pre># Example for Community rules<br /># url = http://www.snort.org/pub-bin/downloads.cgi/Download/comm_rules/Community-Rules.tar.gz<br /><br /># Example for rules from the Bleeding Snort project<br /># url = http://www.bleedingsnort.com/bleeding.rules.tar.gz<br /><br /># Emerging Threats<br />url = http://rules.emergingthreats.net/open/snort-2.9.0/emerging.rules.tar.gz<br /><br /># Example for UNIX.<br /># tmpdir = /home/oinkmaster/tmp/<br />tmpdir = /tmp</pre><br />Create a directory to backup old rules when Oinkmaster updates rules:<br /><br /><code># mkdir /tmp/rules.old</code><br /><br />Run the Oinkmaster perl script for the first time:<br /><br /><code># oinkmaster.pl -o /etc/snort/rules/ -b /tmp/rules.old -C oinkmaster.conf</code><br /><br /><ul><li>-o outputs the downloaded rules to the specified directory</li><li>-b defines where to backup old rules</li><li>-C defines which configuration file to use.</li><li><b>NOTE</b>: <code>oinkmaster.pl</code> uses the <code>wget</code> utility, install it if it is not present on the system.</ul><br />Output from above command:<br /><br /><pre># ./oinkmaster.pl -o /etc/snort/rules/ -b /tmp/rules.old -C oinkmaster.conf<br />Loading /root/oinkmaster-2.0/oinkmaster.conf<br />Downloading file from http://rules.emergingthreats.net/open/snort-2.9.0/emerging.rules.tar.gz... done.<br />Archive successfully downloaded, unpacking... done.<br />Setting up rules structures... done.<br />Processing downloaded rules... disabled 0, enabled 0, modified 0, total=15999<br />Setting up rules structures... done.<br />Comparing new files to the old ones... done.<br />Updating local rules files... done.<br /><br />[***] Results from Oinkmaster started 20110915 20:06:35 [***]<br /><br />[*] Rules modifications: [*]<br /> None.<br /><br />[*] Non-rule line modifications: [*]<br /> None.<br /><br />[+] Added files (consider updating your snort.conf to include them if needed): [+]<br /><br /> -> BSD-License.txt<br /> -> classification.config<br /> -> compromised-ips.txt<br /> -> emerging.conf<br /> -> emerging-activex.rules<br /> -> emerging-attack_response.rules<br /> -> emerging-botcc.rules<br /> -> emerging-chat.rules<br /> -> emerging-ciarmy.rules<br /> -> emerging-compromised.rules<br /> -> emerging-current_events.rules<br /> -> emerging-deleted.rules<br /> -> emerging-dns.rules<br /> -> emerging-dos.rules<br /> -> emerging-drop.rules<br /> -> emerging-dshield.rules<br /> -> emerging-exploit.rules<br /> -> emerging-ftp.rules<br /> -> emerging-games.rules<br /> -> emerging-icmp.rules<br /> -> emerging-icmp_info.rules<br /> -> emerging-imap.rules<br /> -> emerging-inappropriate.rules<br /> -> emerging-malware.rules<br /> -> emerging-misc.rules<br /> -> emerging-mobile_malware.rules<br /> -> emerging-netbios.rules<br /> -> emerging-p2p.rules<br /> -> emerging-policy.rules<br /> -> emerging-pop3.rules<br /> -> emerging-rbn-malvertisers.rules<br /> -> emerging-rbn.rules<br /> -> emerging-rpc.rules<br /> -> emerging-scada.rules<br /> -> emerging-scan.rules<br /> -> emerging-shellcode.rules<br /> -> emerging-smtp.rules<br /> -> emerging-snmp.rules<br /> -> emerging-sql.rules<br /> -> emerging-telnet.rules<br /> -> emerging-tftp.rules<br /> -> emerging-tor.rules<br /> -> emerging-trojan.rules<br /> -> emerging-user_agents.rules<br /> -> emerging-virus.rules<br /> -> emerging-voip.rules<br /> -> emerging-web_client.rules<br /> -> emerging-web_server.rules<br /> -> emerging-web_specific_apps.rules<br /> -> emerging-worm.rules<br /> -> gen-msg.map<br /> -> gpl-2.0.txt<br /> -> rbn-ips.txt<br /> -> rbn-malvertisers-ips.txt<br /> -> reference.config<br /> -> sid-msg.map<br /> -> snort-2.9.0-open.txt<br /> -> unicode.map</pre><br />Edit <code>/etc/snort/snort.conf</code>:<br /><br /><ul><li>Now add the emerging.conf configuration file as an include.</li></ul><br /><pre>###################################################<br /># Step #7: Customize your rule set<br /># For more information, see Snort Manual, Writing Snort Rules<br />#<br /># NOTE: All categories are enabled in this conf file<br />###################################################<br /><br /># Emerging Threats rules<br />include $RULE_PATH/emerging.conf</pre><br />By default <code>/etc/snort/rules/emerging.conf</code> has all rule groups commented out, and they must be enabled manually.<br /><br />Edit <code>/etc/snort/rules/emerging.conf</code> to enable specific Emerging Threats rules:<br /><br /><ul><li>Ensure classification.config and reference.config are uncommented!</li><li>Note do <i>not</i> enable all rules, you should enable rules relevant to your environment.</li><li>As an example I enabled the rules shown below.</li></ul><br /><pre>include $RULE_PATH/classification.config<br />include $RULE_PATH/reference.config<br /><br />include $RULE_PATH/emerging-policy.rules<br />include $RULE_PATH/emerging-games.rules<br />include $RULE_PATH/emerging-virus.rules<br />include $RULE_PATH/emerging-attack_response.rules<br />include $RULE_PATH/emerging-icmp.rules<br />include $RULE_PATH/emerging-scan.rules<br />include $RULE_PATH/emerging-chat.rulesles<br />include $RULE_PATH/emerging-shellcode.rules<br />include $RULE_PATH/emerging-current_events.rules<br />include $RULE_PATH/emerging-malware.rules<br />include $RULE_PATH/emerging-worm.rules<br />include $RULE_PATH/emerging-misc.rules<br />include $RULE_PATH/emerging-exploit.rules<br />include $RULE_PATH/emerging-p2p.rules<br />include $RULE_PATH/emerging-mobile_malware.rules<br /><br />include $RULE_PATH/emerging-botcc.rules<br />include $RULE_PATH/emerging-compromised.ruless<br />include $RULE_PATH/emerging-drop.rulesK.rules<br />include $RULE_PATH/emerging-dshield.rules<br />include $RULE_PATH/emerging-rbn.rules<br />include $RULE_PATH/emerging-rbn-malvertisers.rules</pre><br />Take time to go through each file to determine whether or not to use the rulesets provided by Emerging Threats. Again, not all will be relevant to your environment.<br /><br />Because we edited <code>/etc/snort/rules/emerging.conf</code>, we need to ensure that it is not accidentally "updated" or overwritten by Oinkmaster. Remember, Oinkmaster outputs to and updates the <code>/etc/snort/rules</code> directory.<br /><br />Edit <code>oinkmaster-2.0/oinkmaster.conf</code> again:<br /><br /><pre>#######################################################################<br /># Files to totally skip (i.e. never update or check for changes) #<br /># #<br /># Syntax: skipfile filename #<br /># or: skipfile filename1, filename2, filename3, ... #<br />#######################################################################<br /><br /># Do not modify or update emerging.conf<br />skipfile emerging.conf</pre><br /><b>NOTE</b>: You can optionally ignore the rulesets that are commented out as well, i.e. <code>skipfile emerging-tor.rules,emerging-sql</code> etc.<br /><br />Now anytime you need to update your Snort rules, simply run the <code>./oinkmaster.pl -o /etc/snort/rules/ -b /tmp/rules.old -C oinkmaster.conf</code> command. It is recommended to put this command in a daily cron job, as Emerging Threats updates their rules on a daily basis.<br /><br /><font style="font-size: 140%; color: #424242; weight: bold;">Running Snort</font><br /><br />Finally, run Snort in the foreground with the following command:<br /><br /><code># snort -c /etc/snort/snort.conf -A fast -b -i br0</code><br /><br /><ul><li>-A specifies the alert mode.</li><li>-b will log packets that were alerted on in tcpdump format.</li><li>-i specifies what interface to listen on.</li></ul><br />Alerts will be logged to <code>/var/log/snort/alert</code>, if using the fast alert mode, alerts will appear as a single line. <br /><br />Example output of <code>/var/log/snort/alert</code> using Snort alert-mode fast:<br /><br /><pre>09/15-20:55:08.642022 [**] [1:2012141:2] ET POLICY Protocol 41 IPv6 encapsulation potential 6in4 IPv6 tunnel active [**] [Classification: Potential Corporate Privacy Violation] [Priority: 1] {IPV6-ICMP} fe80:0000:0000:0000:0000:5efe:c0a8:0bf1 -> fe80:0000:0000:0000:0000:5efe:181c:c109<br />09/15-21:01:55.234549 [**] [1:2003089:4] ET GAMES STEAM Connection (v2) [**] [Classification: Potential Corporate Privacy Violation] [Priority: 1] {TCP} 1.2.3.4:49158 -> 68.142.72.250:27031<br />09/15-21:02:22.628590 [**] [1:2012170:2] ET GAMES Blizzard Web Downloader Install Detected [**] [Classification: Potential Corporate Privacy Violation] [Priority: 1] {TCP} 1.2.3.4:49196 -> 12.129.206.133:1119</pre><br />I recommend running Snort for at least a day or two, you are guaranteed to see several alerts especially if your sensor is facing an external public network.<br /><br /><font style="font-size: 140%; color: #424242; weight: bold;">Tuning alerts</font><br /><br />After running Snort for a period of time, even for a day or two as previously mentioned, you can quickly get a baseline of the alerts on your network. Some of these alerts can occur quite frequently and may unnecessarily flood your log file. Ideally you want to reduce the amount of "noise" in the alert log file. Tuning your alerts either by disabling the rule signature entirely, or suppressing them using the <code>threshold.conf</code> file is one of the many best practices in maintaining and optimizing Snort.<br /><br /><font style="font-size: 120%; color: #424242; weight: bold;">Disabling an alert</font><br /><br />For example, on my development sensor I saw several of the following alerts:<br /><br /><pre>09/16-17:39:08.567824 [**] [1:2012170:2] ET GAMES Blizzard Web Downloader Install Detected [**] [Classification: Potential Corporate Privacy Violation] [Priority: 1] {TCP} 1.2.3.4:54741 -> 173.223.52.219:80<br />09/16-17:39:08.677677 [**] [1:2012170:2] ET GAMES Blizzard Web Downloader Install Detected [**] [Classification: Potential Corporate Privacy Violation] [Priority: 1] {TCP} 1.2.3.4:54741 -> 173.223.52.219:80<br />09/16-17:39:13.197098 [**] [1:2012170:2] ET GAMES Blizzard Web Downloader Install Detected [**] [Classification: Potential Corporate Privacy Violation] [Priority: 1] {TCP} 1.2.3.4:54742 -> 12.129.242.21:80</pre><br />As seen above, Snort detected a Blizzard Web Downloader which was triggered from a desktop running World of Warcraft within my network. If this were a corporate network, company policy would likely dictate that employees should not be playing games during work. However, because this development Snort sensor is on my home network, this alert is completely harmless and therefore can simply be disabled.<br /><br />The signature or rule ID that triggered the above alert is <code>2012170</code>. The <code>2</code> denotes that this is the second revision of the rule. <br /><br />Using <code>grep</code> to search for <code>2012170</code> within the <code>/etc/snort/rules</code> directory, the file that contains the rule can be found:<br /><br /><pre># cd /etc/snort/rules<br /># grep -n 2012170 *<br /><br />emerging-games.rules:326:alert tcp $HOME_NET any -> $EXTERNAL_NET any (msg:"ET GAMES Blizzard Web Downloader Install Detected"; flow: established,to_server; content: "User-Agent|3a| Blizzard Web Client"; nocase; classtype:policy-violation; sid:2012170; rev:2;)<br />sid-msg.map:11036:2012170 || ET GAMES Blizzard Web Downloader Install Detected</pre><br />The command found the rule in <code>emerging-games.rules</code> on line 326. To disable the rule, simply comment it out with <code>#</code>, like so:<br /><br /><pre>#emerging-games.rules:326:alert tcp $HOME_NET any -> $EXTERNAL_NET any (msg:"ET GAMES Blizzard Web Downloader Install Detected"; flow: established,to_server; content: "User-Agent|3a| Blizzard Web Client"; nocase; classtype:policy-violation; sid:2012170; rev:2;)</pre><br />Restart Snort and the rule ID will no longer trigger an alert to the <code>/var/log/snort/alert</code> log file.<br /><br /><b>IMPORTANT</b>: The <code>oinkmaster.conf</code> still needs to be configured to ensure that the above rule we disabled <i>stays</i> disabled during an update.<br /><br />Edit your <code>oinkmaster.conf</code>:<br /><br /><ul><li>Use a <code>disablesid</code> statement to disable the rule.</li></ul><br /><pre>########################################################################<br /># SIDs to comment out, i.e. disable, after each update by placing a #<br /># '#' in front of the rule (if it's a multi-line rule, it will be put #<br /># in front of all lines). #<br /># #<br /># Syntax: disablesid SID #<br /># or: disablesid SID1, SID2, SID3, ... #<br />########################################################################<br /><br />disablesid 2012170 # ET GAMES Blizzard Web Downloader Install Detected</pre><br />The configuration file allows for comments to be appended to the <code>disablesid</code> parameter. For reference, putting the description of the rule, or why the rule is being disabled is strongly recommended.<br /><br />Now, during an Oinkmaster update the above signature will stay disabled.<br /><br /><font style="font-size: 120%; color: #424242; weight: bold;">Suppressing an alert</font><br /><br />There may be scenarios, where you do not want to disable a rule entirely, but rather configure it so that it does not alert on specific factors, such as source or destination IP addresses.<br /><br />Take for instance the following alerts that were lightly flooding my alert file:<br /><br /><pre>09/16-12:20:43.317651 [**] [1:1390:6] GPL SHELLCODE x86 inc ebx NOOP [**] [Classification: Executable Code was Detected] [Priority: 1] {TCP} 74.125.239.6:80 -> 1.2.3.4:54066<br />09/16-12:20:43.329227 [**] [1:1390:6] GPL SHELLCODE x86 inc ebx NOOP [**] [Classification: Executable Code was Detected] [Priority: 1] {TCP} 74.125.239.9:80 -> 1.2.3.4:54229<br />09/16-12:20:43.365007 [**] [1:1390:6] GPL SHELLCODE x86 inc ebx NOOP [**] [Classification: Executable Code was Detected] [Priority: 1] {TCP} 74.125.224.170:80 -> 1.2.3.4:54222</pre><br />Resolving the above IP addresses shows that they actually belong to Google:<br /><br /><pre># for i in 74.125.239.6 74.125.239.9 74.125.224.170; do whois -h asn.shadowserver.org origin $i; done<br />[Querying asn.shadowserver.org]<br />[asn.shadowserver.org]<br />15169 | 74.125.239.0/24 | GOOGLE | US | GOOGLE.COM | GOOGLE INC<br />[Querying asn.shadowserver.org]<br />[asn.shadowserver.org]<br />15169 | 74.125.239.0/24 | GOOGLE | US | GOOGLE.COM | GOOGLE INC<br />[Querying asn.shadowserver.org]<br />[asn.shadowserver.org]<br />15169 | 74.125.224.0/24 | GOOGLE | US | GOOGLE.COM | GOOGLE INC</pre><br />After further investigation it is safe to assume that the alerts were false positives. Optionally the triggering rule can be disabled, as shown in the previous example above, however, doing so would allow <i>any</i> IP address meeting the requirements to alert signature ID <code>1390</code> to pass through the network undetected. The correct way to tune the alert is to ignore only if it originates from Google, which can be done using the <code>/etc/snort/threshold.conf</code> file.<br /><br />Configure <code>/etc/snort/threshold.conf</code> to ignore alerts from signature ID <code>1390</code> if it originates from Google's networks:<br /><br /><pre># Suppression:<br />#<br /># Suppression commands are standalone commands that reference generators and<br /># sids and IP addresses via a CIDR block (or IP list). This allows a rule to be<br /># completely suppressed, or suppressed when the causitive traffic is going to<br /># or comming from a specific IP or group of IP addresses.<br />#<br /># Suppress this event completely:<br />#<br /># suppress gen_id 1, sig_id 1852<br />#<br /># Suppress this event from this IP:<br />#<br /># suppress gen_id 1, sig_id 1852, track by_src, ip 10.1.1.54<br />#<br /># Suppress this event to this CIDR block:<br />#<br /># suppress gen_id 1, sig_id 1852, track by_dst, ip 10.1.1.0/24<br />#<br /><br /># Suppress [1:1390:6] GPL SHELLCODE x86 inc ebx NOOP [**] if originating from Google's networks.<br />suppress gen_id 1, sig_id 1390, track by_src, ip 74.125.0.0/16</pre><br />The Snort developers were kind enough to provide different examples in the<code>threshold.conf</code> file.<br /><br />Lastly, restart Snort and the the signature ID will only alert on non-Google networks.<br /><br />Nothing needs to be changed on the Oinkmaster side, as no rule was explicitly being disabled or enabled. Suppression is handled by <code>threshold.conf</code> which is outside of the <code>/etc/snort/rules</code> directory that Oinkmaster "tracks".<br /><br /><font style="font-size: 140%; color: #424242; weight: bold;">Resources</font><br /><br />Snort Official Documention: <a href="http://www.snort.org/docs">http://www.snort.org/docs</a><br />Emerging Threats Rule Documentation Wiki: <a href="http://doc.emergingthreats.net/">http://doc.emergingthreats.net/</a>Rodolf Sabalburohttp://www.blogger.com/profile/06648192806453188759noreply@blogger.com3tag:blogger.com,1999:blog-8509744489347868336.post-19436376969334364262011-09-01T21:03:00.001-07:002011-09-15T03:18:26.087-07:00Network analysis of malware infected PCSo I took a short break from work early this morning to run a small errand. I decided to check my email on my desktop computer before I headed back out. Man was I in for a surprise. I moved the mouse cursor to wake the monitor and I was greeted by a nice little program that looked like this:<br /><br /><a href="http://www.spywarevoid.com/wp-content/uploads/2011/08/SecurityProtection.jpg" onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}"><img style="float:left; margin:0 10px 10px 0;cursor:pointer; cursor:hand;width: 550px; height: 300px;" src="http://www.spywarevoid.com/wp-content/uploads/2011/08/SecurityProtection.jpg" border="0" alt="" /></a><br /><br />"Security Protection" my ass. My desktop was infected with malware and I'm a network and systems engineer wanting to move into the realm of information security and information assurance. I feel like a dentist getting a cavity or a police officer getting pulled over for speeding.<br /><br />I was caught with my pants down...at my ankles...while bending over. See <a href="http://nooooooooooooooo.com/">http://nooooooooooooooo.com/</a> for my initial reaction upon seeing the above image. I don't think I've been infected by malware in close to 4-5 years.<br /><br />My "check my email for before I head out" quickly became "figure out what the hell happened and block your desktop from connecting back to out to the Internet, and fix it when you get home."<br /><br />I remembered that no more than a month ago I had installed Snort 2.9.0.5 inline (between my cable modem and embedded x86 ALIX router running Linux) on an old Pentium III desktop to vet my CentOS 6.0 32-bit hardened build for work. Cool, I thought, maybe I can analyze the packet captures, log files, and alerts and see what else I can find.<br /><br />Alas, my poor desktop PC, became a case study for incident response and network analysis of a malware infected PC.<br /><br />It was almost 9:30 AM by this time, and I had to head back to work so I came up with a game plan:<br /><br />1. Check Snort logs to see if anything was alerted.<br />2. Block all traffic from the entire local subnet reaching out to the Internet, log it, and see<br />what shows up in the logs.<br />3. Try to run a quick diagnostics and scans on infected desktop before I head back to work.<br />4. Analyze everything further and decide how I will re-mediate my desktop.<br /><br /><font style="font-size: 140%; weight: bold; color: #424242;">Game Plan Item #1: check Snort logs</font><br /><br />I had been running Snort in the background with <code>snort -c /etc/snort/snort.conf -A fast -b -K pcap -i br0</code> on and off for the past two weeks to get an idea of what kind of alerts I would see over time.<br /><br />A quick look at /var/log/snort/alerts, the default log file for Snort alerts, and I saw a couple suspicious entries:<br /><br /><pre>09/01-02:33:13.716611 [**] [1:2012609:2] ET CURRENT_EVENTS Java Exploit Attempt Request for .class from octal host [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} X.X.X.X:54027 -> 76.74.152.98:8080<br />09/01-02:33:13.716718 [**] [1:2011582:8] ET POLICY Vulnerable Java Version 1.6.x Detected [**] [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} X.X.X.X:54028 -> 76.74.152.98:8080<br /></pre><br />The good news was Snort did detect something malicious going on; I didn't have time to check out it further until later.<br /><br /><font style="font-size: 140%; weight: bold; color: #424242;">Game Plan Item #2: block local egress traffic</font><br /><br />I had to cage this beast quickly before it downloaded even more malware and did more damage. I went with a quick rule in Shorewall on my Linux router to block (and log) the entire local subnet from reaching the Internet until I got home.<br /><br />/etc/shorewall/rules:<br /><br /><pre># BLOCK!<br />DROP:info loc net<br /></pre><br />Shorewall is a front-end for iptables; the actual iptables command would have been <code>iptables -I FORWARD 0 -s 192.168.11.0/24 -j DROP</code>.<br /><br />I restarted Shorewall, went to grab a drink, and actually started to do my errands. You know, what I originally had intended to do when I made a quick stop home.<br /><br />These were a couple entries in /var/log/messages:<br /><br /><pre>Sep 1 08:57:42 voyage localadmin: Shorewall restarted<br />Sep 2 09:04:14 voyage kernel: [2084722.015845] Shorewall:loc2net:DROP:IN=eth0 OUT=eth2 SRC=192.168.11.10 DST=65.55.94.220 LEN=52 TOS=0x00 PREC=0x00 TTL=127 ID=429 DF PROTO=TCP SPT=49192 DPT=443 WINDOW=8192 RES=0x00 SYN URGP=0 MARK=0x2<br />Sep 2 09:04:17 voyage kernel: [2084725.097856] Shorewall:loc2net:DROP:IN=eth0 OUT=eth2 SRC=192.168.11.10 DST=65.55.94.220 LEN=52 TOS=0x00 PREC=0x00 TTL=127 ID=430 DF PROTO=TCP SPT=49192 DPT=443 WINDOW=8192 RES=0x00 SYN URGP=0 MARK=0x2<br />Sep 2 09:04:23 voyage kernel: [2084731.247840] Shorewall:loc2net:DROP:IN=eth0 OUT=eth2 SRC=192.168.11.10 DST=65.55.94.220 LEN=48 TOS=0x00 PREC=0x00 TTL=127 ID=431 DF PROTO=TCP SPT=49192 DPT=443 WINDOW=8192 RES=0x00 SYN URGP=0 MARK=0x2<br /><br />Sep 2 09:06:13 voyage kernel: [2084841.657851] Shorewall:loc2net:DROP:IN=eth0 OUT=eth2 SRC=192.168.11.10 DST=65.55.53.156 LEN=52 TOS=0x00 PREC=0x00 TTL=127 ID=440 DF PROTO=TCP SPT=49193 DPT=443 WINDOW=8192 RES=0x00 SYN URGP=0 MARK=0x2<br />Sep 2 09:06:16 voyage kernel: [2084844.741827] Shorewall:loc2net:DROP:IN=eth0 OUT=eth2 SRC=192.168.11.10 DST=65.55.53.156 LEN=52 TOS=0x00 PREC=0x00 TTL=127 ID=441 DF PROTO=TCP SPT=49193 DPT=443 WINDOW=8192 RES=0x00 SYN URGP=0 MARK=0x2<br />Sep 2 09:06:22 voyage kernel: [2084850.890854] Shorewall:loc2net:DROP:IN=eth0 OUT=eth2 SRC=192.168.11.10 DST=65.55.53.156 LEN=48 TOS=0x00 PREC=0x00 TTL=127 ID=442 DF PROTO=TCP SPT=49193 DPT=443 WINDOW=8192 RES=0x00 SYN URGP=0 MARK=0x2<br /></pre><br />As you can see I restarted Shorewall at 8:57 AM, and at 9:04 AM my infected desktop (192.168.11.10) was trying to tunnel out via SSL (TCP 443) to destination address 65.55.94.220! It obviously failed 3 times to reach that address, so it tried again but a different IP at 65.55.53.156. I had an idea of what was to come, so I went on to my next plan of action.<br /><br /><font style="font-size: 140%; weight: bold; color: #424242;">Game Plan Item #3: attempt diagnostics</font><br /><br />Since I cut it off from the Internet, I tried to characterize the behavior of the malware and its effect it had on my desktop. I didn't click anything on the Security Protection Center window, I wanted to kill the process directly.<br /><br />Open up task manager to try to kill process directly? The window automatically closed.<br />Open up command prompt from Start > Run? The window automatically closed.<br />Open up anti-virus software? The window automatically closed.<br />Open up anti-spyware software? The window automatically closed.<br /><br />Well that was characterized quite quickly.<br /><br />I headed back to work and had an excited yet anxious feeling during my shift. On one hand my desktop had malware on it, on the other hand I had Snort packet captures and log files of real malware I could analyze; something I've always wanted to do.<br /><br /><font style="font-size: 140%; weight: bold; color: #424242;">Game Plan Item #4: additional analysis</font><br /><br />When I got home, the first thing I did was check out what new log entries had appeared since I started blocking traffic from loc -> net. Since I normally don't log traffic from loc -> net, a simple grep command on my Linux router would filter my exact results: <code>grep Shorewall:loc2net /var/log/messages</code>. Let's just say there were A LOT of log entries.<br /><br />I did a word count on my grep command to tell me exactly how many log entries there were:<br /><br /><pre># grep loc2net messages | wc -l<br />3869<br /></pre><br />So between approximately 9:00 AM this morning and 6:00 PM, there were close to 4000 denied attempts from my desktop to reach other malicious hosts on the Internet. That's some persistent malware, to say the least.<br /><br />I ran the following command to get an idea of how many of those destination IPs were unique:<br /><br /><pre># grep loc2net messages | awk '{print $10}' | sort -rn | uniq -c<br /><br /> 3 DST=91.209.196.169<br /> 4 DST=83.145.197.2<br /> 14 DST=74.54.61.194<br /> 95 DST=74.125.53.188<br /> 6 DST=74.125.53.141<br /> 9 DST=74.125.53.132<br /> 9 DST=74.125.53.125<br /> 34 DST=74.125.47.108<br /> 15 DST=74.125.239.9<br /> 39 DST=74.125.239.8<br /> 48 DST=74.125.239.7<br /> 42 DST=74.125.239.6<br /> 48 DST=74.125.239.5<br /> 48 DST=74.125.239.4<br /> 39 DST=74.125.239.3<br /> 6 DST=74.125.239.2<br /> 6 DST=74.125.239.15<br /> 6 DST=74.125.239.14<br /> 6 DST=74.125.239.13<br /> 12 DST=74.125.239.12<br /> 8 DST=74.125.239.11<br /> 6 DST=74.125.239.10<br /> 12 DST=74.125.239.1<br /> 12 DST=74.125.239.0<br /> 6 DST=74.125.224.82<br /> 51 DST=74.125.224.239<br /> 36 DST=74.125.224.238<br /> 42 DST=74.125.224.237<br /> 39 DST=74.125.224.236<br /> 24 DST=74.125.224.235<br /> 30 DST=74.125.224.234<br /> 48 DST=74.125.224.233<br /> 66 DST=74.125.224.232<br /> 84 DST=74.125.224.231<br /> 72 DST=74.125.224.230<br /> 90 DST=74.125.224.229<br /> 99 DST=74.125.224.228<br /> 81 DST=74.125.224.227<br /> 84 DST=74.125.224.226<br /> 75 DST=74.125.224.225<br /> 69 DST=74.125.224.224<br /> 9 DST=74.125.224.191<br /><br />...additional output omitted...<br /><br /> 3 DST=69.171.228.14<br /> 12 DST=69.171.224.67<br /> 5 DST=66.235.120.98<br /> 105 DST=66.220.146.22<br /> 3 DST=65.55.94.220<br /> 3 DST=65.55.94.216<br /> 21 DST=65.55.53.190<br /> 3 DST=65.55.53.156<br /> 18 DST=65.55.27.219<br /> 24 DST=65.55.25.59<br /> 9 DST=65.55.200.156<br /> 3 DST=65.55.200.155<br /> 9 DST=65.55.200.139<br /> 12 DST=65.55.184.16<br /> 9 DST=65.55.184.152<br /> 4 DST=65.55.17.39<br /> 9 DST=65.55.119.90<br /> 6 DST=65.54.75.98<br /> 9 DST=65.54.75.95<br /> 9 DST=65.54.75.93<br /> 15 DST=65.54.75.92<br /> 15 DST=65.54.75.8<br /> 3 DST=65.54.75.71<br /> 18 DST=65.54.75.6<br /> 3 DST=65.54.75.51<br /> 3 DST=65.54.75.40<br /> 3 DST=65.54.75.25<br /><br />...additional output omitted...<br /><br /> 49 DST=24.28.193.9<br /> 12 DST=24.25.230.9<br /> 45 DST=24.25.230.8<br /> 42 DST=24.25.230.18<br /> 30 DST=24.25.230.16<br /> 9 DST=24.25.230.10<br /> 4 DST=217.149.52.196<br /> 84 DST=216.228.124.39<br /> 63 DST=216.18.194.133<br /> 3 DST=216.156.213.179<br /> 3 DST=216.156.213.152<br /> 3 DST=213.35.100.25<br /> 4 DST=209.62.68.168<br /> 15 DST=209.18.46.99<br /> 15 DST=209.18.46.91<br /> 6 DST=209.18.46.66<br /> 3 DST=209.18.46.50<br /> 6 DST=209.18.46.42<br /> 3 DST=209.18.46.123<br /> 3 DST=209.151.233.98<br /> 28 DST=208.49.52.91<br /> 26 DST=208.49.52.106<br /> 1 DST=208.43.217.90<br /> 12 DST=207.46.21.123<br /> 10 DST=207.171.162.56<br /> 2 DST=207.171.162.142<br /> 36 DST=204.160.114.254<br /> 3 DST=199.66.201.169<br /> 3 DST=199.47.217.174<br /> 35 DST=199.47.217.173<br /> 30 DST=199.47.217.144<br /> 3 DST=178.255.83.0<br /> 9 DST=173.223.52.217<br /> 9 DST=173.223.52.186<br /> 12 DST=107.20.249.120<br /></pre><br />I piped the above grep command through word count to give me an exact number:<br /><br /><pre># grep loc2net messages | awk '{print $10}' | sort -rn | uniq | wc -l<br />163<br /></pre><br />That makes 163 unique IP addresses my malware infected desktop was trying to contact, and on many of those IPs, they were trying to be contacted several times.<br /><br />I decided to check out those Snort alerts in further detail.<br /><br />I accumulated roughly 642 Snort alerts in /var/log/snort/alerts in the past week. There were a bunch of false positives that were triggered by my PC games, IMs, etc. A quick and dirty regex filtered out all that and gave me what I really needed to see:<br /><br /><pre># cd /var/log/snort<br /># egrep "X.X.X.X:.* -> .*" alert | egrep -v "GAME|P2P|Skype|Google" | less -S<br /></pre><br />Output from the above command:<br /><br /><pre>08/30-02:27:54.042284 [**] [1:2012607:2] ET USER_AGENTS Lowercase User-Agent header purporting to be MSIE [**] Classification: A Network Trojan was Detected] [Priority: 1] {TCP} X.X.X.X:33852 -> 70.37.68.225:80<br />08/30-02:29:11.933137 [**] [1:2012607:2] ET USER_AGENTS Lowercase User-Agent header purporting to be MSIE [**][Classification: A Network Trojan was Detected] [Priority: 1] {TCP} X.X.X.X:54918 -> 70.37.68.225:80<br />09/01-02:32:38.984298 [**] [1:2010937:2] ET POLICY Suspicious inbound to mySQL port 3306 [**] [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 116.255.188.165:6000 -> X.X.X.X:3306<br />09/01-02:33:13.716611 [**] [1:2011582:8] ET POLICY Vulnerable Java Version 1.6.x Detected [**] [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} X.X.X.X:54027 -> 76.74.152.98:8080<br />09/01-02:33:13.716611 [**] [1:2012609:2] ET CURRENT_EVENTS Java Exploit Attempt Request for .class from octal host [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} X.X.X.X:54027 -> 76.74.152.98:8080<br />09/01-02:33:13.716718 [**] [1:2011582:8] ET POLICY Vulnerable Java Version 1.6.x Detected [**] [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} X.X.X.X:54028 -> 76.74.152.98:8080<br />09/01-02:33:13.716718 [**] [1:2012609:2] ET CURRENT_EVENTS Java Exploit Attempt Request for .class from octal host [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} X.X.X.X:54028 -> 76.74.152.98:8080<br />09/01-02:33:13.716987 [**] [1:2012609:2] ET CURRENT_EVENTS Java Exploit Attempt Request for .class from octal host [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} X.X.X.X:54026 -> 76.74.152.98:8080<br />09/01-02:33:13.717114 [**] [1:2012609:2] ET CURRENT_EVENTS Java Exploit Attempt Request for .class from octal host [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} X.X.X.X:54029 -> 76.74.152.98:8080<br />09/01-02:33:13.717287 [**] [1:2012609:2] ET CURRENT_EVENTS Java Exploit Attempt Request for .class from octal host [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} X.X.X.X:54024 -> 76.74.152.98:8080<br />09/01-02:33:13.717449 [**] [1:2012609:2] ET CURRENT_EVENTS Java Exploit Attempt Request for .class from octal host [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} X.X.X.X:54025 -> 76.74.152.98:8080<br />09/01-02:33:13.900829 [**] [1:2012609:2] ET CURRENT_EVENTS Java Exploit Attempt Request for .class from octal host [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} X.X.X.X:54033 -> 76.74.152.98:8080<br />09/01-02:33:13.901049 [**] [1:2012609:2] ET CURRENT_EVENTS Java Exploit Attempt Request for .class from octal host [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} X.X.X.X:54032 -> 76.74.152.98:8080<br />09/01-02:33:13.901278 [**] [1:2012609:2] ET CURRENT_EVENTS Java Exploit Attempt Request for .class from octal host [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} X.X.X.X:54031 -> 76.74.152.98:8080<br />09/01-02:33:13.901574 [**] [1:2012609:2] ET CURRENT_EVENTS Java Exploit Attempt Request for .class from octal host [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} X.X.X.X:54030 -> 76.74.152.98:8080<br />09/01-02:33:13.907570 [**] [1:2012609:2] ET CURRENT_EVENTS Java Exploit Attempt Request for .class from octal host [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} X.X.X.X:54034 -> 76.74.152.98:8080<br />09/01-02:33:13.907705 [**] [1:2012609:2] ET CURRENT_EVENTS Java Exploit Attempt Request for .class from octal host [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} X.X.X.X:54035 -> 76.74.152.98:8080<br />09/01-02:36:19.794010 [**] [1:2011894:8] ET TROJAN TDSS/TDL/Alureon MBR rootkit Checkin [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} X.X.X.X:54071 -> 178.238.233.154:80<br />09/01-02:36:55.080334 [**] [1:2003579:4] ET MALWARE Findwhat.com Spyware (clickthrough) [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} X.X.X.X:54215 -> 206.123.102.103:80<br />09/01-02:37:12.889356 [**] [1:2003579:4] ET MALWARE Findwhat.com Spyware (clickthrough) [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} X.X.X.X:54323 -> 206.123.102.103:80<br />09/01-02:37:13.026983 [**] [1:2003579:4] ET MALWARE Findwhat.com Spyware (clickthrough) [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} X.X.X.X:54323 -> 206.123.102.103:80<br />09/01-02:37:35.670005 [**] [1:2012609:2] ET CURRENT_EVENTS Java Exploit Attempt Request for .class from octal host [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} X.X.X.X:54480 -> 109.236.82.46:80<br />09/01-02:37:35.670716 [**] [1:2012609:2] ET CURRENT_EVENTS Java Exploit Attempt Request for .class from octal host [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} X.X.X.X:54482 -> 109.236.82.46:80<br />09/01-02:37:36.505659 [**] [1:2012609:2] ET CURRENT_EVENTS Java Exploit Attempt Request for .class from octal host [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} X.X.X.X:54484 -> 109.236.82.46:80<br />09/01-02:37:36.508068 [**] [1:2012609:2] ET CURRENT_EVENTS Java Exploit Attempt Request for .class from octal host [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} X.X.X.X:54485 -> 109.236.82.46:80<br />09/01-02:37:41.788934 [**] [1:2012612:4] ET TROJAN Hiloti Style GET to PHP with invalid terse MSIE headers [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} X.X.X.X:54489 -> 205.234.129.135:80<br />09/01-02:37:41.997022 [**] [1:2012612:4] ET TROJAN Hiloti Style GET to PHP with invalid terse MSIE headers [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} X.X.X.X:54490 -> 205.134.252.251:80<br /></pre><br />I'm unsure as to whether the initial alerts on August 30 from 2:27-2:29 AM are related or have something to do with the alerts that started firing off on September 1st. Regardless, the bulk of it started at 2:32 AM on September 1st.Rodolf Sabalburohttp://www.blogger.com/profile/06648192806453188759noreply@blogger.com0tag:blogger.com,1999:blog-8509744489347868336.post-28958924985053278632010-01-14T13:28:00.000-08:002010-01-14T13:32:47.183-08:00Linux hyperterminal for Cisco routers and switchesUsing the Linux screen command you can communicate with Cisco routers and switches through the console port.<br /><br />First verify what serial device was detected (usually S0):<br /><br />$ dmesg | tail<br /><br />Then connect using screen:<br /><br />$ screen /dev/ttyS0<br /><br />I've tested this on Cisco 2811 ISRs, Aironet's, and Catalyst 2900 series switches.Rodolf Sabalburohttp://www.blogger.com/profile/06648192806453188759noreply@blogger.com0tag:blogger.com,1999:blog-8509744489347868336.post-43594962060176090022009-11-28T15:43:00.000-08:002009-11-28T17:50:30.114-08:00How-to get rid of the netbook remix interface in Ubuntu Netbook Remix 9.10 (Karmic)This is quick work around to get rid of the netbook remix interface under Karmic Koala by creating a new user with the default GNOME desktop configuration settings.<br /><br />On your current user remove the ubuntu-netbook-remix-default-settings package, this will remove the main package as well:<br /><blockquote>sudo aptitude purge ubuntu-netbook-remix-default-settings</blockquote>Create a new user using the GUI: System > Administration > Users and Groups > Add User<br /><br />Switch User (DO NOT Log Out) to the new user that was just created. The default Gnome desktop should now be loaded.<br /><br />Certain startup applications need to be disabled. Go to System > Preferences > Startup Applications, uncheck 'Maximus Windows Management' and 'Netbook Launcher'<br /><br />Switch User again to your original user and reinstall the ubuntu-netbook-remix-default-settings package to preserve the netbook interface.<blockquote>sudo aptitude install ubuntu-netbook-remix-default-settings</blockquote>Now you can use the original user for the netbook remix interface, or switch to the new user that was created to use the original Gnome desktop.Rodolf Sabalburohttp://www.blogger.com/profile/06648192806453188759noreply@blogger.com0