FreeBSD 7.0, PostgreSQL 8.3.5 , sed and fkeys on upgrade schema/data of intranet portal

PostgreSQL 8.3.5 on amd64-portbld-freebsd7.0, compiled by GCC cc (GCC) 4.2.1 20070719  [FreeBSD]

FreeBSD 7.0-RELEASE #0: Wed May 21 10:13:51 UTC 2008

in short, the task was to migrate the development database schema to a production one with the production data  ..

1. you do separated backup – per schema and data.
2. you split the the schema file in two parts  – schema only with no fkeys,  and only fkeys ..
3. create database
4. import the schema file with no fkeys
5. populate  the data
6. import the fkeys only file
7. never sign a rent contract in Mataró when you dont understand a word  in Catalan
8. its done, it took me 6 minutes to import half a million records , which is not much

the issue I had was the fkeys
solution : use sed .. i just love sed … 10 years I do use sed awk and bash for system administration and so far no perl at all
cat backup.schema.2009-12-05.00.07.00.sql | sed ‘/^ALTER TABLE ONLY/, /^ *ADD CONSTRAINT .* FOREIGN KEY/!d’ > fkeys.only.sql
cat backup.schema.2009-12-05.00.07.00.sql | sed ‘/^ALTER TABLE ONLY/, /^ *ADD CONSTRAINT .* FOREIGN KEY/d’ > schema.without.fkeys.sql

hope this will save someone ass in upgrading 500 mln db …:)
I do have 120 mln db in fact … its just statistics , and thank to pgsql .. it works like a charm
I have 10 euro until the 1-st Jan 2010, thank to my so called optimism …
i promise this to be my last risky step i take here in Spain. Because I risk my family in fact.
New Year – new luck …
and … frackem them all … So say we all!

Currently listening “Whitesnake – Crying in the Rain“

Slackware ,lighttpd-1.5.0.cache , mod_proxy, fastcgi and pound loadbalancer

In short,

I use pound for one time switch to hot copy backup servers in case some critical services at the primary box are down or with considerable delay.
Nice yeah ?:)
But I needed a box to install mailing software for a friend of mine.
This particaular software – 12all uses at install time the method OPTIONS , and seems it does not work with pound redirection logic.
So I decided to set the lighttpd at the front at the port 80 and to move pound to listen at 8080.
At the lighttpd config I set all the php files to be served the spawned fatscgi php, along with some other directories like xcache admin and some status handlers.
ALL the rest is set to be handled by pound, and the balancing works like a charm, no issues at all, pound redirects the proper requests to the backend servers, and all the garbage XXX injection is dropped by 503 header ..

Slackware 4life

P.S. The VPS host at San Jose http://www.vr.org/ is one of the very few offering Slackware at VPS … thank you gentlemen.

swf2avi conversion project

SWF2AVI – stable on Slackware 12.2,
source: swf + xml + pics info via mysql : result : movie with codec Microsoft MPEG-4 V2  ,

conversion via oficial Adobe Flash player 10,0,32,18
project considered closed

CentOS 5 lighttpd-1.5.0.cache No package ‘lua’ found

checking for LUA… configure: error: Package requirements (lua >= 5.1) were not met:
If you are here, anyway you tried to compile lighttpd –with-lua
and if you got this one:
No package ‘lua’ found,
the soliution is bellow:

root@lb1:[Mon Oct 12 21:04:48]:[/opt/installs/lighttpd-1.5.0.cache]$ wget  http://www.lua.org/ftp/lua-5.1.4.tar.gz
tar xvfz lua-5.1.4.tar.gz
cd lua-5.1.4
make linux
make install
cp etc/lua.pc /usr/lib/pkgconfig/
and you are ready to try again

cable extender RS232 RJ45 Sound D-Sub 9-pin Panasonic, LG, Mitshubishi, Samsung

Well, it took me two years or so to master the cables I do for a project of mine: http://digisign-bg.com
Issue: you have big mama TV set away from the PC player, so you need video and sound signals to get  from the PC to the TV.

Good: Most of the modern TV sets have VGA and sound slots available.
Bad   : You need cable extenders and those at the market  most of the the time are useless – high price, bad quality or even non working at all.
Bad : wireless transfer is sexy, expensive  but useless at long distance and in home environment with a lot of interference.

So , I went a bit further, thanks to the good luck I got  LG TV set. Why good luck? Because  almost ALL of the LG brand have RS232 CONTROL  interface along with VGA and the SOUND.
And as I read the manuals for real, at LG they  use bisybox to listen to RS232  interface … i.e. LG is Linux powered
My respects to them.
Anyway, after short googling for the available options it was clear to me that one have to buy expensive professional extenders in a view to use the TV set as decent movie player.

So, Samsung, Panasonic, Mitsubishi use staight RS232 cables, LG does  not.
I made so far VGA to RJ45  extenders,  tested up to 60 meters , NO issues at all.
I made RS232 with Sound to RJ45 extenders for Panasonic, LG, Mitsubishi, Samsung – no issues at all up to 100 meters.
Have to say I tested with FTP cable. Not UTP, because the istallations I do are important and I use only high quality cables.
Will test next month VGA  and RS2332 with stereo sound extender  over UTP of 200 meters, when I get some cash to byu so long cable.
At the end, I can play a movie at the TV from the Slackware box of my daughter at the LG LCD, to switch to TV channels whenever I want, to mute the tv, control the volume,  to switch it off or on .. etc, and all that from Barcelona, Spain, while my daughter is still is Sofia , Bulgaria …:/

I am starting the cable-extenders.com , for custom/hand  made cable extenders. And will post there my project for embeded linux device with http API for controlling different brands of TV sets. Because  all of the popular TV brands  use different commands, protocols, checksums etc crap, as its high security issues someone to be able to use external control devices to manage the TV :/

Note, the www.cable-extenders.com is not published yet,
I host simple loadbalancer at www.cable-extenders.com in a view to avoid the idiot bots injecting porn hosts at the web server logs.

to be continued ..

CentOS 5 at VPS, postfix master.cf no valid IP address found: smtp

In short,

some of the VPS providers do not let you have eth0 or loopback interfaces.
and when you configure the postfix to send only mode:

root@lb1:[Mon Oct 12 14:53:21]:[/var/adm]$ cat /etc/postfix/main.cf
# CentOS
queue_directory = /var/spool/postfix
command_directory = /usr/sbin
daemon_directory = /usr/libexec/postfix
mail_owner = postfix
myhostname = lb1.cable-extenders.com
mydomain = lb1.cable-extenders.com
myorigin = $mydomain
inet_interfaces = loopback-only
unknown_local_recipient_reject_code = 550
relayhost =
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
debug_peer_level = 2
debugger_command =
PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin
xxgdb $daemon_directory/$process_name $process_id & sleep 5
sendmail_path = /usr/sbin/sendmail.postfix
newaliases_path = /usr/bin/newaliases.postfix
mailq_path = /usr/bin/mailq.postfix
setgid_group = postdrop
html_directory = no
manpage_directory = /usr/share/man
sample_directory = /usr/share/doc/postfix-2.3.3/samples
readme_directory = /usr/share/doc/postfix-2.3.3/README_FILES

after service postfix start you got something like this in the log:

root@lb1:[Mon Oct 12 14:55:30]:[/var/adm]$ cat maillog | sed ‘1,4!d’
Oct 11 12:03:42 svac101 postfix/postfix-script: starting the Postfix mail system
Oct 11 12:03:42 svac101 postfix/master[27022]: fatal: /etc/postfix/master.cf: line 18: no valid IP address found: smtp
Oct 11 12:09:26 svac101 postfix/postfix-script: starting the Postfix mail system
Oct 11 12:09:26 svac101 postfix/master[29049]: fatal: /etc/postfix/master.cf: line 2: no valid IP address found: smtp

after some search, i got this post here: http://www.linuxmail.info/postfix-multiple-ip-address-smtp-greeting/

so, the master.cf  looks like :
root@lb1:[Mon Oct 12 14:58:12]:[/var/adm]$ cat /etc/postfix/master.cf
# Slackware
#smtp      inet  n       -       n       -       -       smtpd
217.113.247.122:smtp inet n – n – - smtpd -o myhostname=lb1.cable-extenders.com
pickup    fifo  n       -       n       60      1       pickup
cleanup   unix  n       -       n       -       0       cleanup
qmgr      fifo  n       -       n       300     1       qmgr
tlsmgr    unix  -       -       n       1000?   1       tlsmgr
rewrite   unix  -       -       n       -       -       trivial-rewrite
bounce    unix  -       -       n       -       0       bounce
defer     unix  -       -       n       -       0       bounce
trace     unix  -       -       n       -       0       bounce
verify    unix  -       -       n       -       1       verify
flush     unix  n       -       n       1000?   0       flush
proxymap  unix  -       -       n       -       -       proxymap
proxywrite unix -       -       n       -       1       proxymap
smtp      unix  -       -       n       -       -       smtp
relay     unix  -       -       n       -       -       smtp
-o smtp_fallback_relay=
showq     unix  n       -       n       -       -       showq
error     unix  -       -       n       -       -       error
retry     unix  -       -       n       -       -       error
discard   unix  -       -       n       -       -       discard
local     unix  -       n       n       -       -       local
virtual   unix  -       n       n       -       -       virtual
lmtp      unix  -       -       n       -       -       lmtp
anvil     unix  -       -       n       -       1       anvil
scache    unix  -       -       n       -       1       scache
the Slackware header is because i did copy the master.cf from other VPS i have at San Jose and its Slackware ..
its much better that this CentOS box because I have full control, including eth0, loopback device and iptables  …
So you comment the line
smtp      inet  n       -       n       -       -       smtpd
and add this one : your_real_IP:smtp inet n – n – - smtpd -o myhostname=your_domain_name

Slackware 12.2 lighttpd-1.5.0-r1992 http sqf balanacing , php spawn-fcgi and nagios-2.3.1

I discovered lighttpd several months ago, and just recently had chance to implement it as http proxy balancer for a group of 4 DL3205Gp boxes.
I started with the 1.4.3, but  finished with 1.5.0-r1992 …
All the boxes has lighty and Apache running, I have one box as http proxy  balancing the rest, and in case its dead, one of the resting three will become the loadbalancer.  The balancing is done via lighttpd 1.5 and its mod_proxy_core and mod_proxy_backend_http modules.
Have to say, I use one of the boxes as NFS server,  and the resting three  mount the webserver directory in a view to serve the same contect …
The tricky part was to bypass the nagios url from balancing, for abious reasons.
The trick: in lighttpf.conf
$HTTP["url"] !~ “\.(js|css|gif|jpg|png|ico|txt|swf|html|htm)$|^\/(nagios|nagadmin)” {
#$HTTP["url"] =~ “\.php$” {
proxy-core.protocol = “http”
proxy-core.balancer = “sqf”
proxy-core.allow-x-sendfile = “enable”
proxy-core.backends = (
#  lighty pool
# dont use the box “127.0.0.1:80″
“10.99.99.11:80″
,”192.168.11.12:80″
,”192.168.11.13:80″
#  APACHE pool
#  dont use apache at the hosting box …,”127.0.0.1:8800″
,”10.99.99.11:8800″
,”192.168.11.12:8800″
,”192.168.11.13:8800″

)
proxy-core.max-pool-size = 3000
}

in the light we have include of nagios.conf:
alias.url =     (
“/nagios/cgi-bin” => “/usr/local/nagios/sbin”,
“/nagios” => “/usr/local/nagios/share”
)

$HTTP["url"] =~ “^/nagios/cgi-bin” {
cgi.assign = ( “” => “” )
}

$HTTP["url"] =~ “nagios” {
auth.backend = “htpasswd”
auth.backend.htpasswd.userfile = “/etc/nagios/.htpasswd”
auth.require = ( “” => (
“method” => “basic”,
“realm” => “Monitoring area”,
“require” => “valid-user”
)
)
setenv.add-environment = ( “REMOTE_USER” => “user” )
$HTTP["url"] =~ “\.php$” {
proxy-core.balancer = “round-robin”
proxy-core.protocol = “fastcgi”
proxy-core.allow-x-sendfile = “enable”
proxy-core.backends = ( “unix:/tmp/php-fastcgi.sock”)
proxy-core.max-pool-size = 16
}
}

In this way the nagios url is handled by the local box only.
same settings will be for interna box, but using the proxy-core http protocol  instead …

At the end, I would like to thank Carlos Silva Santin for giving me  the chance to wotk with him in the past 30 days ..
Carla  will grow up so fast Carlos … :p

Slackware 12.2 pure-ftpd v1.0.22 425 Sorry, invalid address given

add -H /dont resolve/ option to your start line

OLD:
/usr/local/sbin/pure-ftpd -o -E -l puredb:/etc/pureftpd.pdb -l unix  -p 40000:50000 &
NEW:
/usr/local/sbin/pure-ftpd -o -E -l puredb:/etc/pureftpd.pdb -l unix  -p 40000:50000 -H &

Cheers

Slackware 12.2 and MGE Ellipse MAX 600

In case you got the geeky   MGE Ellipse MAX 600 , you have two cables, one usb and one serial
use the USB one, in my cause I used one spare USB extender 5 m long, because the slack box i use atm is a bit far from the MGE
anyway, I use the UPS to power the phone line, the ADSL WiFi modem and one dd-wrt powered Linksys WRT54GL

will skip the reason why, but I added one slack box to the circuit.
the box controls one 37′ LG LCD TV, sound, RGB input, power off power on, everything. anyway, in short.

install network_ups_tools-2.4.1-i486-1an
edit /etc/ups.conf:
[mge]
driver = usbhid-ups
port = auto
vendorid = 0463
desc = “MGE UPS Systems UPS”
edit /etc/upsd.conf
MAXAGE 15
LISTEN 127.0.0.1 3493
MAXCONN 1024
edit /etc/upsmon.conf
RUN_AS_USER root
MONITOR mge 1 upsmon s31k master
MINSUPPLIES 1
SHUTDOWNCMD “/sbin/shutdown -h +0″
POLLFREQ 5
POLLFREQALERT 5
HOSTSYNC 15
DEADTIME 15
POWERDOWNFLAG /etc/killpower
NOTIFYFLAG ONLINE       SYSLOG
NOTIFYFLAG ONBATT       SYSLOG
NOTIFYFLAG LOWBATT      SYSLOG
NOTIFYFLAG FSD  SYSLOG
NOTIFYFLAG COMMOK       SYSLOG
NOTIFYFLAG COMMBAD      SYSLOG
NOTIFYFLAG SHUTDOWN     SYSLOG
NOTIFYFLAG REPLBATT     SYSLOG
NOTIFYFLAG NOCOMM       SYSLOG
NOTIFYFLAG NOPARENT     SYSLOG
RBWARNTIME 43200
NOCOMMWARNTIME 300
FINALDELAY 5
edit /etc/upsd.users
[root]
password = pass
actions = SET
instcmds = ALL
[upsmon]
password  = pass
upsmon master
—————————————-
touch /etc/rc.d/rc.upsd
chmod 0700 /etc/rc.d/rc.upsd
edit /etc/rc.d/rc.upsd :
checkMGE=”$(/sbin/lsusb | grep “MGE UPS Systems UPS” | grep -v grep)”
if [ "${checkMGE}" != "" -a "$(ps ax | grep /usr/bin/usbhid-ups | grep -v grep)" == "" ]
then
/usr/bin/upsdrvctl start mge
if [ -f /var/state/ups/usbhid-ups-mge.pid ]
then
if [ "$(file /var/state/ups/usbhid-ups-mge | grep socket | grep -v grep)" != "" -a "$(ps ax | grep /usr/sbin/upsd | grep -v grep)" == ""  ]
then
/usr/sbin/upsd
fi
fi
elif [ "${checkMGE}" != "" -a "$(ps ax | grep /usr/bin/usbhid-ups | grep -v grep)" != "" ]
then
if [ -f /var/state/ups/usbhid-ups-mge.pid ]
then
if [ "$(file /var/state/ups/usbhid-ups-mge | grep socket | grep -v grep)" != "" -a "$(ps ax | grep /usr/sbin/upsd | grep -v grep)" == ""  ]
then
/usr/sbin/upsd
fi
fi
else
echo “No need to start anything …”
fi

/usr/bin/upslog -s mge -l /var/log/ups.log -f “%TIME @Y-@m-@d @H:@M:@S% %VAR battery.charge% %VAR input.voltage% %VAR ups.load% [%VAR ups.status%] %VAR ups.temperature% %VAR input.frequency%”
/usr/sbin/upsmon -p

then add this at the end of the rc.local:
upsdctl=”/etc/rc.d/rc.upsd”
if [ -x "${upsdctl}" ]
then
${upsdctl}
fi
————————————————————————
this is all, you will have the logs updated every 30 seconds :
==> ups.log <==
2009-04-03 03:08:57 92 NA 11 [OL CHRG] NA NA
in a view to check the current MGE status just run:
upsc mge
example:
root@burgas-mol:[Fri Apr 03 03:09:38]:[/var/adm]$ upsc mge
battery.charge: 92
battery.charge.low: 20
battery.runtime: 1914
battery.type: PbAc
driver.name: usbhid-ups
driver.parameter.pollfreq: 30
driver.parameter.pollinterval: 2
driver.parameter.port: auto
driver.parameter.vendorid: 0463
driver.version: 2.4.1
driver.version.data: MGE HID 1.12
driver.version.internal: 0.34
input.sensitivity: normal
input.transfer.boost.low: 185
input.transfer.high: 285
input.transfer.low: 165
input.transfer.trim.high: 265
input.voltage.extended: no
outlet.1.desc: PowerShare Outlet 1
outlet.1.id: 2
outlet.1.status: on
outlet.1.switchable: no
outlet.desc: Main Outlet
outlet.id: 1
outlet.switchable: no
output.frequency.nominal: 50
output.voltage: 230.0
output.voltage.nominal: 230
ups.beeper.status: enabled
ups.delay.shutdown: 20
ups.delay.start: 30
ups.firmware: 1.01
ups.load: 11
ups.mfr: MGE UPS SYSTEMS
ups.model: Ellipse MAX 600
ups.power.nominal: 600
ups.productid: ffff
ups.serial: PbAc
ups.status: OL CHRG
ups.timer.shutdown: -1
ups.timer.start: -1
ups.vendorid: 0463

Slackware 12.0 Postgresql 8.3.7 and slony1-2.0.1 replication engine

in short, i had like 5 hours spent in useless efforts to figure out why slony is not compiling at production box while at my desktop box the compile ran flawlessly
short solution:

root@darkwater:[Fri Mar 20 02:45:16]:[/opt/installs/slony/slony1-2.0.1]$ cat ../seik.slony1-2.0.1.sonf
#!/bin/sh
cd /opt/installs/slony/slony1-2.0.1
export LDFLAGS=-ldl
make clean
./configure  –with-pgconfigdir=/opt/pgsql/bin –prefix=/opt/slony2
sed -i ’s/LDFLAGS += $(rpath)/LDFLAGS += $(rpath) -ldl/’ Makefile.global
#-with-docs
make -j3 install

for ppl willing to learn, the issue was in the missing -ldl flag first in the default slackware 12.0 LDFLAGS
i dont know what is the reason for this,
At Slackware 12.2.0 slony 2.0.1 compile ran without single issue….

anyway, i added a fix in the Makefile.global, so make now compiles the slony instance …
for ppl unaware of sed .. LEARN sed and awk 

seikath