Sourcepole - Aktuell

spam acl condition: error reading from spamd socket: Connection reset by peer

2010-07-17T12:45:00Z

Occassionaly we’ve been seeing messages like:

spam acl condition: error reading from spamd socket: Connection reset by peer

in /var/log/exim4/paniclog

But lately the problem has become much more persistent. I found out, that there’s some arsehole spammer sending us 500K spams every ca. 70 minutes. From /var/log/mail.log:

Jul 17 08:52:32 mail spamd[477]: spamd: identified spam (7.9/5.0) for spamd:1004 in 288.5 seconds, 505669 bytes.

Now since spamassassin would take that long (nearly 5 minutes!!!) to find out whether that mail is a spam or not, exim would simply timeout it’s connection to spamassassin and the result was the message seen above.

What I did was to use sa-compile to compile the SA rules to “native code” and to enable “Rule2XSBody”, which on Debian lives in /etc/spamassassin/v320.pre.

This seems to make SA quite a lot faster and to use less CPU.

If this won’t help we could also restrict the maximum size of messages to scan.

Yours Tomáš Pospíšek

FOSS4G 2010 Barcelona

2010-06-22T07:39:00Z

Die internationale Open Source GIS-Konferenez FOSS4G findet dieses Jahr vom 6.-9. September 2010 in Barcelona statt.

Sourcepole ist mit drei Vorträgen vertreten:

Comparison of Open Source Virtual Globes
Performance and statistical analysis of WMS servers
SpatiaLite, the Shapefile of the future?

Zusätzlich unterstützen wir den QGIS-Workshop und nehmen mit dem QGIS-Mapserver an WMS Performance Shootout teil.

Translation of Twinkle SIP phone presentation

2010-06-04T00:08:00Z

Since I found Michel de Boer’s explanation of many of the fundamental SIP concepts in his 2005 dutch presentation of the Twinkle SIP phone very short and easy to follow - contrary to a lot of other material about SIP out there - I’ve translated it to english.

Here it is: Twinkle - a SIP softphone for Linux

Graphing ping times

2010-06-03T23:52:00Z

Well, sometimes one wonders why people write tools that wheight a megabyte, when it’s easy to stick together one or two tools with a script and have the same thing.

Here’s a ruby script that generates a ping latency graph (target is google):

#!/usr/bin/ruby
#
# based on
#   http://oss.oetiker.ch/rrdtool/prog/rrdruby.en.html example
# with help of the tutorial
#   http://oss.oetiker.ch/rrdtool/tut/rrdtutorial.en.html
# and the docu
#   http://oss.oetiker.ch/rrdtool/doc/rrdcreate.en.html
# 
# placed in the public domain by Tomáš Pospíšek

require "RRD"

$rrd    = `mktemp /tmp/tmp.XXXXXXXX`.chomp
$grafik = `mktemp /tmp/tmp.XXXXXXXX`.chomp
$start  = 0

def now()
  Time.now.to_i
end

def create()
  $start=now()
  RRD.create(
    $rrd,
    "--start", "#{$start - 1}",
    "--step", "#{1}",       # every 1s
    "DS:pinglatency:GAUGE:10:0:200", # 10s of missing data are OK, min=0, max=200 (ping time)
    "RRA:AVERAGE:0.5:1:#{60*30}") # 60 seconds * 30 minutes
  puts "RRD database created #{$rrd} at #{$start}"
  puts "Your graph is at #{$grafik}"
end

def ping_latency
   `ping -c 1 -n -w 1 google.com` =~ /.* time=(\d+)/
  # 64 bytes from 74.125.43.147: icmp_seq=1 ttl=52 time=31.6 ms
  return $1
end

def update()
  puts "updating #{$rrd} with #{now()}:#{ping_latency()}"
  RRD.update($rrd, "#{now()}:#{ping_latency()}")
end

def generate_gfx()
  # puts "generating graph #{$grafik}"
  RRD.graph(
   "#{$grafik}",
    "--title", " Ping latency to Google",
    "--x-grid", "SECOND:10:MINUTE:1:MINUTE:2:0:%X",
    "--start", $start.to_s,
#    "--end", # -> now
    "--interlace",
    "--imgformat", "PNG",
    "--width=450",
    "DEF:pinglatency=#{$rrd}:pinglatency:AVERAGE",
    "AREA:pinglatency#00b6e4:ms")
end

#main
create()
while(true)
  update()
  generate_gfx()
  sleep 1
end

It generates a graph like this:

Folien QGIS User Meeting Bern

2010-04-23T08:26:00Z

Das erste deutschsprachige QGIS User Meeting in Bern war mit rund 80 Teilnehmern ein grosser Erfolg.

Die Folien der Präsentation des OpenLayers-Plugin sind hier erhältlich.

Die gesammelten Präsentation werden auf der QGIS-Homepage veröffentlicht.

Referate FOSSGIS 2010

2010-03-08T14:15:00Z

Sourcepole war an der deutschsprachigen FOSSGIS-Konferenz 2010 in Osnabrück gleich mit drei Vorträgen vertreten.

Die Präsentationsunterlagen können hier heruntergeladen werden:

Die detaiilierte Anleitung für den WMS-Performance Test kann im OSGEO-Wiki nachgelesen werden.

Besten Dank für das sehr positive Feedback!

QGIS User Meeting Bern 21.4.10

2010-03-05T10:23:00Z

Am 21. April 2010 findet das erste deutschsprachige QGIS User-Meeting in Bern statt. Das Programm und die detaillierten Angaben können auf der QGIS-Homepage nachgelesen werden.

Sourcepole stellt bei dieser Gelegenheit ihr OpenLayers-Plugin vor, das Google Maps und Konsorten als QGIS-Layer ins Desktop-GIS integriert.

Alle QGIS-Anwender und Interessierte sind herzlich zu diesem Anlass eingeladen!

Testing complex Webapps with wget

2010-02-24T20:25:00Z

How do you test web applications? You can use Selenium, but Selenium is very heavy.

I tried wget.

To find out what exactly to send to the app, I started Firebug inside Firefox, went to the “Network” section and recorded what I would do, to see, whether the application works. Then I took the accessed URLs together with the parameters and first tried to send them manually from the command line. Observe, that Firebug lets you copy the exact POST/GET parameter as was sent by the browser, when you right click on the POST/GET command in the Firebug Network log. Once I managed to redo all the steps manuall from the commandline I transferred what I did into a script and edited it until it was nice and would run through:

$ history > test_my_web_app

test_my_web_app now looks as follows:

#!/bin/bash
#
# check availability of:
url="http://www.example.org/my-app" 

# exit immediately if something goes wrong -> test fails!
set -e

echo "Testing $url" 

# save an error log in case there's an error so that
# we can analyse it further
error_log=$( mktemp )
cookies=$(   mktemp )

echo "In case the tests fail, you'll find the error log under $error_log" 
# send everything that goes to STDERR to the log
exec 2>$error_log

echo "Getting a cookie..." 
wget --save-cookies $cookies "$url" -O /dev/null

# we could put username/password directly into the script but we're cautuous and don't
echo -n "please enter user for $url: ";     read    user
echo -n "please enter password for $url: "; read -s pass
echo

echo "Logging in..." 
# if we logged in successfully, then the webapp will send us an html page
# that contains the following string:
to_verify="You're logged in" 
# POST to the webapp:
wget --load-cookies $cookies \
     --post-data "login=Login&user_login=$user&user_password=$pass" \
     --save-cookies $cookies \
     --keep-session-cookies \
     "$url/login" \
     -O - \
| grep -q "$to_verify" 

# now we're logged in

echo "Query component DB..." 
# First search the DB (parameters need to be URI encoded...)
num_components_found=$(
  wget --load-cookies $cookies \
       --save-cookies $cookies \
       --post-data 'action=set_params&crit%5Bcolor%5D=red&crit%5Bmin_size%5D=0' \
       "$url/prepare-picture" \
       -O - \
  | grep 'Number of matching components:' \
  | sed 's/.*Number of matching components:<\/td><td>//; s/<.*//' )

echo "Make sure that the app found something..." 
[ $num_components_found -gt 0 ]

echo "Generate and get picture..." 
wget --load-cookies $cookies \
     --save-cookies $cookies \
     --post-data 'index=0' \
     "$url/get-pic" \
     -O /dev/null

echo "Hooray, $url successfully tested" 

rm $error_log
rm $cookies
<pre>

Surprisingly easy, isn't it?

FOSSGIS 2010

2010-02-23T09:29:00Z

Auch dieses Jahr ist Sourcepole an der deutschsprachigen FOSSGIS-Konferenz vertreten. Die FOSSGIS 2010 findet vom 2.-5. März 2010 in Osnabrück statt.

Sourcepole ist mit drei Vorträgen vertreten:

Vorinformation: Die internationale Open Source GIS-Konferenez FOSS4G findet dieses Jahr vom 6.-9. September 2010 in Barcelona statt.

munin plugin multi_http_responsetime doesn't recognize dead servers

2010-02-01T09:38:00Z

Here’s a patch:

# diff -u multi_http_responsetime.orig multi_http_responsetime
--- multi_http_responsetime.orig        2010-02-01 10:13:22.000000000 +0100
+++ multi_http_responsetime     2010-02-01 10:30:23.000000000 +0100
@@ -16,6 +16,9 @@
 use Time::HiRes qw( time );
 use IO::Socket;

+$FAILED=0;
+$INFINITE=999999;
+
 # ----- config -----
 push(@url_array, "http://www.google.de");
 push(@url_array, "http://www.t-online.de");
@@ -34,7 +37,7 @@
                                        PeerPort => 80,
                                        Proto => 'tcp'
                                        );
-       return(0) unless ($sock);
+       return($FAILED) unless ($sock);
        print $sock "GET $baseurl HTTP/1.1\nHost: $vhost\nConnection: close\n\n";
        while (<$sock>) {
                $data .= $_;
@@ -63,11 +66,16 @@
                $baseurl =~ s@^\w+://.+?(/)@\1@;

                $tick1 = time();
-               geturl;
+               $retval = geturl;
                $tick2 = time();

-               $tspent = $tick2-$tick1;
-               $msecs = ($tspent * 1000);
+               if ($retval == $FAILED) {
+                       $msecs = $INFINITE;
+                       }
+               else {
+                       $tspent = $tick2-$tick1;
+                       $msecs = ($tspent * 1000);
+                       }

                printf "timespent$i.value %.3f\n", $msecs;
                }

Chopped up SIP

2010-01-23T00:18:00Z

Since upgrading to Karmic I had the problem under Ubuntu that SIP calls under Twinkle would start get all chopped up every few minutes.

Turns out it’s NetworkManager that scans the wireless spectrum every two minutes for networks even if you are allready connected to some AP.

While people seem to be fixing the problem in the proper way, by making scanning interfere less with an active connection, there’s a good soul that provides patched versions of NetworkManager that gives up scanning while there’s an active connection to an AP.

See https://bugs.launchpad.net/ubuntu/+source/network-manager/+bug/373680

Rack::Monitor

2010-01-19T16:56:00Z

Rack::Monitor is Rack middleware collecting run-time information for monitoring tools like Munin.

It can be integrated in any Rack based appication. Sample configuration for Ruby on Rails:

 config.middleware.use Rack::Monitor, :url => '/rack_status', :watch => ['/', '/ping']

The collected statistical values can be accessed at /rack_status. It is recommended to limit access to this path with our contributed middleware Rack::Access from rack-contrib.

Sample munin output:

statuscodes-day”/>

removing gnome-screensaver

2010-01-19T13:54:00Z

In https://bugs.launchpad.net/bugs/509411 I had the problem, that locking the screen would just not work any more. It turned out that gnome-screensaver must be perpetualy runnung in the background in order to be able to lock the screen.

Also this would affect the computer not locking the screen when suspending.

It might be that Gnome/Ubuntu are user-friendly, however the bloat is starting to annoy me. Why do I need to be running 214 processes, when the only thing that is actually being used is firefox and a terminal? Why do I need a process (gnome-screensaver) that is using up CPU cycles and virtual memory, when it’s actually never doing anything useful?

Thus I purged gnome-screensaver (8MB) and installed i3lock.

There are three things missing:

1) Make the laptop lock the screen when suspending:

$ cat /etc/pm/sleep.d/01_screenlock
#!/bin/bash
#
# 01_screenlock: lock screen at suspend with i3lock
# based on http://bbs.archlinux.org/viewtopic.php?id=68158
#

USER=henry
DISPLAY=:0.0
IS_ACTIVE="$( pidof i3lock )"

lock_screen() {
  # skip if i3lock is allready running
  if [ ! "$IS_ACTIVE" ]; then
       su - $USER -c "/bin/sh -c 'DISPLAY=$DISPLAY /usr/bin/i3lock'" &
  fi
}

case $1 in
   hibernate)
       lock_screen
       ;;
   suspend)
       lock_screen
       ;;
   thaw)
       # not required.
       ;;
   resume)
       # not required.
       ;;
   *) exit $NA
       ;;
esac

2) Add it to the panel:

$ cat ~/.gnome2/panel2.d/default/launchers/i3lock.desktop
#!/usr/bin/env xdg-open

[Desktop Entry]
Encoding=UTF-8
Version=1.0
Type=Application
Terminal=false
Icon=/usr/share/icons/gnome/scalable/actions/gnome-lockscreen.svg
Name=Lock the Screen
Exec=i3lock

3) integrate it into package management, so that I don’t accidentaly remove packages on which the screen locking depends:

$ cat control
Section: misc
Priority: optional
Standards-Version: 3.6.2

Package: lock-screen
Maintainer: Tomas Pospisek 
Depends: i3lock, pm-utils, gnome-icon-theme
Files: 01_screenlock /etc/pm/sleep.d/01_screenlock
 i3lock.desktop /home/henry/.gnome2/panel2.d/default/launchers/i3lock.desktop
Description: lock screen on suspend/hybernate
 bla
$ equivs-build control
$ sudo dpkg -i lock-screen_1.0_all.deb

Of course, it’s a hack. However it works and is easy to adapt. Let me know at tpo at sourcepole.ch in case you get to make it more generic.

That’s it. Saved runtime bloat. Saved disk space (i3lock is 60K large).

osgEarth for Ubuntu

2010-01-15T10:55:00Z

osgEarth is a scalable terrain rendering toolkit for OpenSceneGraph.

Our package for Ubuntu is now available at UbuntuGis!

osgviewer sample call:

 OSG_FILE_PATH=/usr/share/osgearth osgviewer --window 50 50 900 700 /usr/share/osgearth/maps/srtm.earth

Press ‘h’ for help. See also OSG Wiki

osgearth_simple sample call:

OSG_FILE_PATH=/usr/share/osgearth osgearth_simple --window 50 50 900 700 /usr/share/osgearth/maps/geopole.earth

Press ‘1’-‘5’ for predefined viewpoints. Drag mouse with left+right click for tilt.

OssimPlanet for Ubuntu

2009-12-02T20:23:00Z

OssimPlanet is an open source project for accurate, high performance, 3D Geo-spatial visualization.

Our package for Ubuntu is now available at UbuntuGis!