Toady's blog

To content | To menu | To search

Friday, December 26 2008

India

I am going to India and I will be away from keyboard until January 11th.

Tuesday, December 23 2008

Nmap facts with parallel coordinates

I played a bit with nmap scans and argus to have a flow-wise graph:

tcpdump -i any -w scan2.pcap -n 'ip'

nmap 192.168.0.11

/usr/sbin/argus -r scan2.pcap -w - | ra -n > scan2.netflow

parsers/net/argus2picviz.pl scan2.netflow > scan2.pgdl

pcv -Tpngcairo scan2.pgdl -Rheatline > scan2-freq.png

Which gives this image: scan2-freq.png

Higher resolution available here.

Doing frequency analysis on those data are quite interesting:

  • We can see that nmap religiously scans the first 1024 ports
  • Among those 1024 ports, some are tested more than others (as we see red lines between source port and dest port) using the same source port
  • Some higher ports are tested several times from different source ports
  • The higher we get on the dest port axis, spreader ports are tested
  • Some localhost tests are tried

Ah and by the way, a lot of Picviz news will arose January 2009, stay tuned!

Tuesday, December 16 2008

udev hell

Beware, this is a rant!


Today I installed a new network card in my machine. My interface that used to be eth0 suddenly got renamed as eth1.

How intuitive! Especially with all the network scripts I have, being a big libpcap user. So what happened?
# dmesg |grep eth0
[    2.672035] forcedeth 0000:00:07.0: ifname eth0, PHY OUI 0x732 @ 1, addr 00:de:ad:be:ef:23
[    2.785500] udev: renamed network interface eth0 to eth1
[    5.460834] eth0: RealTek RTL8139 at 0xffffc20000322000, 00:fe:ed:da:d0:42, IRQ 17
[    5.471601] eth0:  Identified 8139 chip type 'RTL-8100B/8139D'
[   90.021757] eth0_rename: link down
[   90.030918] ADDRCONF(NETDEV_UP): eth0_rename: link is not ready

Yeah, eth0 becomes eth1, eth1 becomes eth0, and eth0 is renamed as eth0_rename. Silly me! how come I am not happy with the current situation?

It is very obvious that you should edit the file "/etc/udev/rules.d/70-persistent-net.rules" and keep only those two lines:
SUBSYSTEM=="net", DRIVERS=="?*", ATTR{address}=="00:de:ad:be:ef:23", NAME="eth0"
SUBSYSTEM=="net", DRIVERS=="?*", ATTR{address}=="00:fe:ed:da:d0:42", NAME="eth1"
Which after a reboot links my card "00:de:ad:be:ef:23" to eth1, and the other to eth0. Very logic huh? ok at least I don't have the "eth0_rename" interface.

I think I will waste less time changing the interface name in my scripts.



Monday, December 15 2008

Picviz lectures tour


I just arrived from San Diego, USA, where I have the opportunity to talk about Picviz and how you can use it to do system logs analysis.
This was part of the Usenix Workshop on the analysis of System Logs. Lectures were interesting, especially those on logs used to do failure prediction. WASL 2008 material is now available on the conference website.

As I already said on the honeynet blog, I went through the Cray log analysis contest. I lost by one vote only, which is fair since I started the contest after my presentation.

My pictures are all uploaded there.

Yesterday, I submitted a paper for the next Eicar convention. This paper is a very maths-centric view of parallel coordinates and of course later talks about how things got implemented in Picviz. I wrote it along with my good friend Philippe. Here's a small teaser:




Sunday, December 7 2008

Usenix slides

My talk is over, and my slides on ' Picviz: finding a needle in a haystack' are available.

Friday, December 5 2008

San Diego

Just arrived, the trip was quite long but worth doing since the weather is pretty cool here.

I walk in streets in shirt and we have clear sunny sky.

Gotta finish my slides now!

Monday, November 10 2008

The ultimate album

Ça fait longtemps que je m'écoute cet album de Fred Couderc, et je dois dire que je ne m'en lasse toujours pas. 

Il faut dire que c'est un album hommage à Roland Kirk, qui en plus d'être un grand maître de la respiration circulaire, était capable de jouer plusieurs instruments à la fois : deux, trois (!) saxophones, ainsi que de la flûte à bec dans le nez pendant qu'il souffle dans la flûte traversière.

Et en dehors du côté spectacle de tout ceci, il n'en restait pas moins capable d'avoir un son et des mélodies remarquables. La barre était donc fixée très haute. Et c'est avec brio que Fred et sa troupe ont sû relever le défi.

Ma préférée est "Phantoms", jouée avec un instrument unique au monde : un C melody droit, fabriqué par mon génial ami Hervé Martin (non ce n'est pas ce sax que vous voyez en photo).

Thursday, November 6 2008

Ulogd2 commiters picvized

Éric wrote a script to parse git commiters and got a nice graph :



I now need to improve the number displaying for next release!

Monday, November 3 2008

firewalls@securityfocus.com dead body :-(

Hi! This is the ezmlm program. I'm managing the
firewalls@securityfocus.com mailing list.

I'm working for my owner, who can be reached
at firewalls-owner@securityfocus.com.

I'm sorry, the list moderators for the firewalls list
have failed to act on your post. Thus, I'm returning it to you.
If you feel that this is in error, please repost the message
or contact a list moderator directly.


If anyone wonders why there's been no messages on the firewalls mailing list of security focus, this email say it all.

Last message is from July 2nd (list archives here).


Monday, October 27 2008

Usenix WASL 08

My paper on Picviz was accepted for the first Usenix Workshop on the Analysis of System Logs:

Picviz: finding a needle in a haystack

When considering log files for security, usual applications available today either look for patterns using signature databases or by using a behavior approach. In both cases, information can be missed. In addition, attackers know how tools work, can discover them and even attack them. The problem becomes bigger when logs are aggregated on a network and several gigabytes of logs are generated per day. Without knowing much about what you are looking for, a parallel coordinates plot, as implemented by Picviz, can visualize correlations making it easy to write tools to automate the recognition.


I am excited to meet smart people in the security arena I have high respect of. And also to enjoy a warm weather in winter ;-)

Tuesday, October 7 2008

OSSIR Bretagne

Avec Pierre nous avons eu l'occasion de donner une présentation à Rennes pour le groupe OSSIR Bretagne. Les diapositives sont disponibles ici.

J'ai eu l'honneur de pouvoir présenter pour la première fois Picviz.

Juste avant nous, il y a eu une présentation de Renaud, qui était excellente : une bonne présentation de ce qu'est la visualisation, comment il s'en servait de façon pratique pour résoudre des problèmes de la vrai vie, une petite pique sur le RAID (forcément ;) et l'utilisation d'Excel pour faire de la correlation graphique !

J'avais un peu peur que nos deux présentations se chevauchent, mais finalement c'était très bien comme cela, j'èspère avoir l'occasion de renouveller l'expérience.


Friday, October 3 2008

TaoSecurity wisdom

"Overall, I felt that a lot of the RAID talks were divorced from operational reality. Several attendees addressed this subject with questions. Too many researchers appear to be working on subjects that would never see the light of day in real networks." - Richard Bejtlich from Taosecurity blog (emphasis is mine).

I can only agree with Richard.

This is exactly why, when searching the ACM digital library for "prelude ids", you keep having papers referring to our work, and none of those 'university ids research centric' conferences accepting our papers.

So I have one message to deliver to several IDS researcher: yes, you use stuff we do, and you are not able to write code nor help the research IDS field because you are simply too far away from the reality.

For those you don't know, RAID stands for Recent Advances in Intrusion Detection Systems, and is supposed to be THE IDS conference, but are unfortunately failing miserably.

Wednesday, October 1 2008

Honeypot fun


Tonight I've been trying to generate a call graph from a malware received on my low interaction honeypot (nepenthes):
[2008-09-28T07:00:25] 218.22.211.45 -> 88.191.82.101 ftp://ip:ip@sky-xunlei.3322.org:21/3389.exe cf0260770c8258f2a8baf0782013614e

Using IDA, I generated a Graph Description Language (GDL) file, and thanks to Perl and its wonderful CPAN, I easily installed a gdl2dot converter:

perl -MCPAN -e shell;
cpan[2]> install TELS/graph/Graph-Easy-0.63.tar.gz


That I then used like this:
graph-easy callgraph.gdl --as_dot > callgraph.dot

And because I prefer to show you a PNG format, I did the conversion:
dot -Tpng callgraph.dot > callgraph.png

Which results on the following graph (clickable image):

There is not much to see now, but I'll collect more of the same malware and using the excellent patchdiff IDA plugin from Tenable to show you differences of coming malware variations here, just be patient :-)





Saturday, September 27 2008

Heatmaps entering picviz trunk

Today, Picviz trunk introduced a new feature: heatmaps.

When a line is drawn, there is currently no way to know how many of the same line were drawn, thus making impossible to separate a line drawn only once and a line drawn hundred times.

The actual implementation is considering a line per event, and when a line between two axes is repeated, the whole line is affected by this frequency.

This is not ideal, I think I will improve this to consider just portions of lines and lines as a whole, instead of a little of both. Anyway, this is a rendering plugin, and you can call the svg plugin to draw it:
pcv -Tsvg -Rheatmap file.pcv > file.svg

On my honeypot data, this draws a picture like this:

Enjoy!

PS: it seems I keep talking about Picviz on this blog; don't worry, honeypots, signatures and IDS are on the way ;)

Saturday, September 20 2008

Picviz 0.3 released

Picviz 0.3 'good coffee' is released. This one mostly focuses on filters, but there is way more stuff in it: read the release notes!

Also, my good friend Pascal Terjan built a package for Mandriva.

Happy urpmi!

Monday, September 15 2008

Filtering with PCRE

I've implemented tonight filtering capabilities using pcre, which allows

pcv -Tsvg -Wpcre samples/test1.pcv 'hide except value = "foo.*" on axis 2'

It is getting complicated, let's separate the two parts :
pcv -Tsvg -Wpcre samples/test1.pcv

and
'show only value = "foo.*" on axis 2'

We have one one side the pcv binary calling the svg plugin (-Tsvg) and asking to use the engine with pcre capabilities (-Wpcre), and on the other the filter saying:

"Please show me any value starting with foo on the third (0,1,2,3..) axis".

Now the question is: shall I let axis 2 be the third axis, or would you prefer 3 ?

Anyway, it is up on svn, enjoy!

Sunday, September 14 2008

Picviz 0.3 beta testing needed

Picviz 0.3 is expected this week.

Lots of things improved, there are 162 commits between the last release and this beta!

Among the new things to test, there are :

  • The 'penwidth' property. Which increases the line width to draw. You can now use it like this:
a="123",b="foobar" [penwidth="0.8"];
of course, you can combine this property with the color:
a="123",b="foobar" [color="#ff0033",penwidth="1.9"];
  • A CSV plugin, which can output your graph as csv file:
toady at marcadet:~/local/scm/svn/picviz/trunk$ pcv -Tcsv samples/test1.pcv 
306,241,137
307,458,127
  • Properties managed with hashes, not a user visible thing, but that makes adding a property a lot easier now
  • PCV tool rewritten, using the same code, so same way tcpdump does with bpf filters, filters explained a bit later
  • Linked list managed differently, again not a user visible thing, just some mess sorted out
  • Filters, *THE filtering feature!* right now, you can only filter know data, such as:
pcv -Tsvg samples/test1.pcv 'show only value = "123" on axis 1'

which will only show values with '123' on the second (1) axis. What I want to add before the release is the filter on the mapped value by the rendering engine,

How to test ? please compile the program, play with it, report comments, bugs etc.. and release will show up very very soon.

Monday, September 8 2008

GCC specs

DindinX vient d'écrire un billet intéressant sur une fonctionnalité de GCC peu connue, les specs permettant de changer le comportement par défaut du compilateur. À lire ici.

Friday, September 5 2008

Netfilter workshop in Paris, 29 September - 3 October 2008

Established since 2001, the Netfilter Workshop (NFWS) serie provides a space for discussing on-going research and development in firewalling for Linux. And this year it is going to happen in Paris.

The program is now online, I am already excited to see Dave talking about multi-TX and the other people. Of course, don't miss Eric's talk on Ulogd2 and Pierre on how to to build a weather-based firewall!

September 29th is the users day, and the entry is free but a registration is asked. Please fill in the following form.

See you there!

Thursday, September 4 2008

Latex hint

To force a figure to appear to a specific place, use [hbt!]

\begin{figure}[hbt!]
\begin{center}
\includegraphics[scale=0.6]{eps/picviz-simplearch.eps}
\end{center}
\caption{Picviz simplified architecture}
\label{fig_picvizsimplearch}
\end{figure}


Thank you dindinx!

- page 1 of 5