30 April 2012

126. linux ssh examples: rsync across portforwarded ssh and helping remotely via ssh behind firewalls

Even bog-standard ssh is pretty neat, since there's little that can't be done in the terminal. However, firewalls can be annoying and if you set somebody up with linux you will have to be prepared to support them for years to come -- at least with debian wheezy the odd breakage happen, and people tend to be less forgiving with linux problems than with windows problems.

Anyway.

1. rsync across a server 
A can connect to B, B can connect to C. A can't connect directly to C. B and C can't connect directly to A. B and C can connect to each other either direction. An example is when B is your home router and C sits on your local network, while A has a public IP but sits behind a corporate firewall.

You want to rsync from A to C

On A, do
ssh user_at_B@B_ip_address -L 5555:C_ip_address:22
then in another terminal
rsync -avz  --progress --stats -e "ssh -p 5555" /home/user_a/work user_c@localhost:/home/user_c/Documents

2. Helping someone in the terminal across a server
A and C can't connect to each-other. A and C can both connect to B. B can't connect to A or C. A wants to connect to C to start e.g a screen session to help out. An example is when both users A and C can connect to a lab router from their respective home, but their ISPs are preventing direct ssh access between them.

Setting up reverse ssh, on C, do
autossh -R 19998:localhost:22 user_B@B_ip_address

Connecting from A to C, do
ssh user_B@B_ip_address -L 19999:localhost:19998
then in another terminal
ssh user_C@localhost -p 19999

You can then set up a screen session to both help and teach.
http://verahill.blogspot.com.au/2012/02/debian-testing-wheezy-64-attach-to.html
http://verahill.blogspot.com.au/2012/03/fun-with-gnu-screen-setting-up-screenrc.html

28 April 2012

125. Fixed: No internet on old Dell after debian testing upgrade

The situation:
I remotely dist-upgraded someone's old Dell laptop (Inspiron 6000). They were running Wheezy, but had not done a full apt-get upgrade/dist-upgrade since January. After a reboot, the laptop would not connect to the internet via either the wired or wireless interfaces.

ip addr and ifconfig showed three ifs:
lo
eth0
eth1

NOTE: A simple reason why you don't have wireless is because you'll need firmware-iwlwifi -- make sure you've enabled non-free and contrib in your repos.

The solution:
Adding
auto eth0
iface eth0 inet dhcp

to /etc/network/interfaces and commenting out
allow-hotplug eth0

followed by executing
sudo service networking restart && sudo service network-manager restart

brought up the wired interface, allowing for installation of packages and remote help. These instructions are simple enough for most to follow and can get you back in the driver's seat.

Next, I installed the wicd packages (wicd-curses, wicd-cli etc.)
sudo apt-get install wicd-cli wicd-curses wicd-daemon python-wicd

 -- at this point about 70 other packages got pulled in. Suspicious. I had seen that apt wanted me to autoremove a lengthy list of programs:

The following packages were automatically installed and are no longer required:  python-crypto mono-2.0-gac python-tagpy libgnomekbd4 seahorse-daemon libboost-python1.46.1 python-pyasn1 libswscale0 libjs-jquery-ui python-twisted-core ekiga python-opengl rpm2cpio xsltproc pnm2ppa libavutil50 gnuchess-book  gnome-nettool gnome-games-extra-data gnome-desktop-data libgnomepanel2.24-cil aisleriot gtali libglade2.0-cil libsushi-1.0-0 python-mako glchess libbabl-0.0-0 gir1.2-javascriptcoregtk-3.0 libgegl-0.0-0 ttf-sil-gentium libicu44  libraw2 libx264-116 libx264-118 tcptraceroute gir1.2-sushi-1.0 gnome-video-effects libcamel-1.2-23 python-clientform gnome-games gnome-session-canberra gdebi anthy-common anthy quadrapassel python-twisted-web libedataserverui1.2-11  update-manager-core gnobots2 libgnome-media0 libqtmultimediakit1 gnome-cards-data liferea-data libgraphite3 libmagickcore4-extra libglew1.6 rhythmbox bsh-gcj python-gdata gedit libnm-util1 xdg-user-dirs-gtk libgexiv2-0  python-gtksourceview2 gnome-office update-manager-gnome libwebkit-1.0-2 python-gtkglext1 xulrunner-5.0 libart2.0-cil libboost-program-options1.46.1 libminiupnpc5 libcamel1.2-19 librpmio2 update-notifier-common libpostproc51 librpm2  seahorse rpm-common libgnome2-perl libaccess-bridge-java-jni xulrunner-8.0 xulrunner-9.0 libnotify1 libaccess-bridge-java update-notifier python-mechanize libwebp0 libgnome2.24-cil libndesk-dbus1.0-cil gnome-media-common libgpod4  gnibbles libavformat52 rhythmbox-data libmono-cairo2.0-cil gedit-plugins libgrilo-0.1-0 zeitgeist-core system-tools-backends libboost-thread1.46.1 libgweather1 libgnome-bluetooth7 dasher python-configobj guile-1.8-libs shotwell  libpolkit-gtk-1-0 libdmapsharing-3.0-2 python-serial gedit-common libboost-serialization1.46.1 libboost-date-time1.46.1 libgmime2.4-cil python-gnomedesktop software-center libopal3.6.8 python-pam python-openssl libhwloc3  libmono-i18n-west2.0-cil libgssdp-1.0-2 simple-scan gir1.2-webkit-3.0 libebook1.2-10 libqtlocation1 python-markupsafe libgck0 python-webkit libmono-posix2.0-cil bsh libedataserverui-3.0-0 rhythmbox-plugin-cdrecorder libgdata11  libhunspell-1.2-0 librhythmbox-core4 librhythmbox-core5 libgtksourceview2.0-common libmono-security2.0-cil libgtksourceview2.0-0 libgnome2-vfs-perl gnotski dasher-data libegroupwise1.2-13 libvpx0 libedata-cal-1.2-11 libnm-glib2  gnome-doc-utils libecal1.2-8 python-twisted-conch libgmime-2.4-2 python-louie libunique-1.0-0 libedataserver1.2-14 libpython2.6 libimobiledevice1 libedata-book-1.2-9 python-nevow gnome-games-data iagno glines media-player-info  python-pysqlite2 libcupsdriver1 python-gnomeapplet libclutter-gtk-0.10-0 gnome-sudoku libslab0a libmagickcore4 tcl gir1.2-gucharmap-2.90 libmono-sharpzip2.84-cil libmagick++4 libmono-corlib2.0-cil python-epsilon  libgnome-window-settings1 libgpod-common rhythmbox-plugins libv8-3.5.10.24 libv8-3.6.6.14 libcryptui0a ntfsprogs python-axiom libmozjs2d libquvi0 python-bugbuddy libmozjs5d gir1.2-rb-3.0 libtracker-client-0.10-0 liferea libpt2.6.7  libmozjs8d libmozjs9d python-coherence libmagickwand4 libchamplain-0.8-1 libgupnp-1.0-3 python-gdbm libchamplain-gtk-0.8-1 gnotravex gnome-netstatus-applet libnl1 libnl3 gnect libavcodec52 python-evolution libanthy0 libyajl1  liboobs-1-4 libgucharmap7 liboobs-1-5 mahjongg libndesk-dbus-glib1.0-cil libid3tag0 libraptor1 gnome-system-tools python-wnck libtracker-sparql-0.10-0 libgnome-desktop-2-17 libtracker-sparql-0.12-0 hamster-applet gnuchess libfreerdp0  libgnome-vfs2.0-cil min12xxw libwebkit-1.0-common minissdpd transmission-common libgtkglext1 libnatpmp1 libnet1 python-twisted-bin libmatroska4 libgupnp-igd-1.0-3 libmono-system2.0-cil libgnome2-canvas-perl file-roller  libxalan2-java-gcj transmission-gtk python-rdflib sound-juicer gnomine

 and suddenly it made sense:

aptitude search gnome-core

pB  gnome-core                                                                                                     - The GNOME Desktop Environment -- essential components


Basically, it seems that gnome got 'uninstalled' (technically just the meta package) -- most of the files related to it won't get removed until you do apt-get autoremove, but it's crippled enough to reduce functionality.
At least that was my thesis.

sudo service network-manager stop
sudo wicd-curses eth1

showed no wireless networks. Something not right.

Make sure the system is really up to date, then reboot, and have another go at it:
sudo apt-get update && sudo apt-get upgrade
which pulled in some glib-networking-* packages.
sudo apt-get install gnome gnome-core
sudo shutdown -r now

After boot
sudo service network-manager stop
sudo wicd-curses eth1

et voilá -- I do get a list over the local wireless networks. There's still a worrying list over stuff when you look at what would be autoremoved (e.g. apt-get -s autoremove)

Rebooted and the wireless interface seems to be handled ok by network-manager.
There is still a number of packages marked for autoremove, so...
sudo apt-get autoremove

System seems to work ok still.
sudo apt-get install update-manager-gnome update-notifier


And my job is done.

Here's the final /etc/network/interfaces:
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
allow-hotplug eth1
#NetworkManager#iface eth0 inet dhcp
#allow-hotplug eth1
auto eth1
#iface eth1 inet dhcp
auto eth0
iface eth0 inet dhcp


23 April 2012

124. gnome-activity-journal and zeitgeist

It may not be advertised widely, but zeitgeist + journal might help save your behind when you're in a hurry and can't find a file.

aptitude show zeitgeist
" Zeitgeist is a service which logs the user's activities and events (files opened, websites visited, conversations held with other people, etc.) and makes the relevant information available to other applications."

And that it does. A service without an interface isn't terribly useful -- so you will want to install gnome-activity-journal as well.


Chances are that zeitgeist is already running on your system, so that you only need to install gnome-activity-journal, which is present in the repos, and then you'll get access to all the old data you've been accumulating.

22 April 2012

123. Adding python support to nwchem under debian

I've posted the general build instructions for nwchem 6.0 with mpi support here: http://verahill.blogspot.com.au/2012/03/nwchem-60-with-openmpi-support-on.html

However, those instructions don't include python support.

0. Download,  extract nwchem and install blas etc. as shown in http://verahill.blogspot.com.au/2012/03/nwchem-60-with-openmpi-support-on.html

1. Edit nwchem-6.0/src/config/makefile.h
Change line 1962 from
EXTRA_LIBS +=    -lnwcutil  -lpthread -lutil -ldl
to
EXTRA_LIBS +=    -lnwcutil  -lpthread -lutil -ldl -lz -lssl
2. Install python headers
sudo apt-get install python2.7-dev

3. Execute the following commands (one by one or by putting them in a shell script)
export LARGE_FILES=TRUE
export TCGRSH=/usr/bin/ssh
export NWCHEM_TOP=`pwd`
export NWCHEM_TARGET=LINUX64
export NWCHEM_MODULES="all python"
export PYTHONHOME=/usr
export PYTHONVERSION=2.7
export USE_MPI=y
export USE_MPIF=y
export MPI_LOC=/usr/lib/openmpi/lib
export MPI_INCLUDE=/usr/lib/openmpi/include
export LIBRARY_PATH=$LIBRARY_PATH:/usr/lib/openmpi/lib
export LIBMPI="-lmpi -lopen-rte -lopen-pal -ldl -lmpi_f77 -lpthread"
cd $NWCHEM_TOP/src
make clean
make nwchem_config
make FC=gfortran

It should work fine and after a long build you'll have python enabled binaries.

4. Testing
You can test whether there's python support by creating test.nw with

python
for n in range(1,6):
        print n, n*2,n**2
end
task python
and running it with 
mpirun -n 1 nwchem test.nw

which gives



  NWChem Input Module
                                -------------------


                               NWChem Python program
                               ---------------------
for n in range(1,6):
        print n, n*2,n**2
1 2 1
2 4 4
3 6 9
4 8 16
5 10 25
1 2 1
2 4 4
3 6 9
4 8 16
5 10 25
 Task  times  cpu:        0.0s     wall:        0.0s

Done.


Note:
for ROCKS/CENTOS it was not necessary to edit src/config/makefile.h


The relevant parts in the the build configuration are
export NWCHEM_MODULES="all python"
export PYTHONHOME=/opt/rocks
export PYTHONVERSION=2.4

Other than that, just follow http://verahill.blogspot.com.au/2012/03/building-nwchem-60-on-rocks-543centos.html



Error:
gfortran: error: /usr/include/python2.7/lib/python2.7/config/libpython2.7.a: No such file or directory
make: *** [all] Error 1
locate libpython2.7.a
/usr/lib/libpython2.7.a
/usr/lib/python2.7/config/libpython2.7.a

Reason:
export PYTHONHOME=/usr/include/python2.7

Solution:
export PYTHONHOME=/usr


Error:
In function `PyZlib_compress':
(.text+0x1540): undefined reference to `deflateEnd'

Solution:
http://www.emsl.pnl.gov/docs/nwchem/nwchem-support/2012/02/0065.Re:_NWCHEM_undocumented_compilation_flag

Edit nwchem-6.0/src/config/makefile.h
For LINUX64 look at lines 1960-1964

1960      ifeq ($(BUILDING_PYTHON),python)
1961 #   EXTRA_LIBS += -ltk -ltcl -L/usr/X11R6/lib -lX11 -ldl
1962      EXTRA_LIBS +=    -lnwcutil  -lpthread -lutil -ldl
1963   LDOPTIONS = -Wl,--export-dynamic
1964      endif

Change line 1962 to
EXTRA_LIBS +=    -lnwcutil  -lpthread -lutil -ldl -lz -lssl


20 April 2012

122. DVB-T in VLC on Debian Wheezy

For some reason me-tv has stopped working properly today  (20/4/2012)-- I can't change the channels, screen size or drop down the menus. In order to quit, you have to kill the application. Something's clearly amiss. (Edit 22/5/2012: seems to be this bug https://bugs.launchpad.net/ubuntu/+source/me-tv/+bug/958751 -- 09/01/2013: it's been fixed now ) kaffeine, although a KDE app, also works well, and is very simple to set up -- the EPG  also works better than vlc.

This is Kaffeine, not VLC
This is me-tv, not VLC


Oh well.

Given that it's Friday night I had to come up with a quick fix.

I've got a Leadtek 1000 DTS, which lspci makes show up as
01:06.0 Multimedia controller: Philips Semiconductors SAA7130 Video Broadcast Decoder (rev 01)

See item 7 in http://verahill.blogspot.com.au/2012/01/debian-testing-64-wheezy-small-fixes.html if your card isn't recognised (Hint: enable PnP in the bios)

So, where's what I did:
1. Install gnome-dvb-*
sudo apt-get install gnome-dvb-daemon gnome-dvb-client vlc

2. Set up your tv card:
gnome-dvb-setup

This bit's easy. Just click your way through. Eventually you'll realize that while gnome-dvb sets up channels ok, it only support recording -- not tv watching. And the whole totem + gnome-dvb-daemon? Not working yet it seems.

3. Start vlc
vlc ~/.config/gnome-dvb-daemon/channels_DVB-T.conf

If all went well you are now watching TV!
You should have a list over channels in your playlist (ctrl+L).
The EPG support could perhaps be better (me-tv is awesome) but you'll find rudimentary listings by going to Tools/Program Guide.

As for me-tv...it might be the same as this bug: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=667731


 Looking at /var/cache/apt/archives I don't see anything obvious:

-rw-r--r-- 1 root root  23232372 Apr 16 05:18 linux-image-3.2.0-2-amd64_3.2.15-1_amd64.deb
-rw-r--r-- 1 root root   3505210 Apr 16 05:18 linux-headers-3.2.0-2-common_3.2.15-1_amd64.deb
-rw-r--r-- 1 root root    572064 Apr 16 05:18 linux-headers-3.2.0-2-amd64_3.2.15-1_amd64.deb
-rw-r--r-- 1 root root    244210 Apr 17 07:32 libtag1-vanilla_1.7.1-2_amd64.deb
-rw-r--r-- 1 root root      9132 Apr 17 07:32 libtag1c2a_1.7.1-2_amd64.deb
-rw-r--r-- 1 root root    195492 Apr 18 03:33 libservlet2.5-java_6.0.35-3_all.deb
-rw-r--r-- 1 root root   1348610 Apr 18 08:18 netpbm_2%3a10.0-15+b1_amd64.deb
-rw-r--r-- 1 root root     94438 Apr 18 08:18 libnetpbm10_2%3a10.0-15+b1_amd64.deb

(I use my own kernel and not the debian one)

Here's my current channel_DVB-T.conf list (Melbourne, Victoria, Australia):

ABC News 24:226500000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_3_4:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:2314:0:560
ABC1:226500000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_3_4:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:512:650:563
ABC2 / ABC4:226500000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_3_4:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:2307:2308:562
ABC3:226500000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_3_4:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:2311:2312:564
ABC Jazz:226500000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_3_4:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:0:2318:567
ABC Dig Music:226500000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_3_4:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:0:2317:566
ABC1:226500000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_3_4:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:512:650:561
GO!:191625000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_3_4:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:517:700:1074
GEM:191625000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_3_4:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:512:0:1073
Nine Digital:191625000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_3_4:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:519:720:1072
EXTRA:191625000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_3_4:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:520:730:1075
ONE:219500000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_3_4:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:514:0:1585
TEN Digital:219500000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_3_4:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:512:650:1589
ONE:219500000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_3_4:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:514:0:1591
ELEVEN:219500000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_3_4:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:516:681:1592
SBS ONE:536625000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_2_3:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_8:HIERARCHY_NONE:161:81:785
SBS HD:536625000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_2_3:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_8:HIERARCHY_NONE:102:103:789
SBS TWO:536625000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_2_3:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_8:HIERARCHY_NONE:162:83:786
SBS 3:536625000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_2_3:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_8:HIERARCHY_NONE:161:81:787
SBS 4:536625000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_2_3:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_8:HIERARCHY_NONE:161:81:788
SBS Radio 1:536625000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_2_3:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_8:HIERARCHY_NONE:0:201:798
SBS Radio 2:536625000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_2_3:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_8:HIERARCHY_NONE:0:202:799

7:177500000:INVERSION_OFF:BANDWIDTH_7_MHZ:FEC_2_3:FEC_2_3:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_8:HIERARCHY_NONE:769:770:1328
7 Two:177500000:INVERSION_OFF:BANDWIDTH_7_MHZ:FEC_2_3:FEC_2_3:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_8:HIERARCHY_NONE:769:770:1330
7 Mate:177500000:INVERSION_OFF:BANDWIDTH_7_MHZ:FEC_2_3:FEC_2_3:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_8:HIERARCHY_NONE:769:770:1331

121. Connection to remote server via an intermediary server

We have three computers: A, B and C

A is our desktop. C is a remote computer we want to connect to. B is a server in the middle.

For various reasons we want to first connect from A to B, then from B to C. A scenario is where C is behind a firewall bridged by B.

A specific scenario is this:
I have a linksys router, e.g. Terbium.internet.com, which manages the network in a lab. There are several computers on the network in the lab behind Terbium: Sulfur, Phosphorous, Oxygen (192.168.1.106) etc.
My home computer is Niobium.

Terbium is a linksys router (W54G) running tomato and you have to log in as root using a keyfile.
Oxygen is a debian box where I have a personal account as user verahill.

The ugly method:
In the same terminal on computer Niobium do
ssh root@terbium.internet.com
ssh verahill@192.168.1.106

This works fine for working remotely on oxygen (192.168.1.106). However, file transfer is a pain

The pretty method:
In one terminal on Niobium, do

ssh root@terbium.internet.com -L 9999:192.168.1.106:22

In another terminal on Niobium, do
ssh verahill@localhost -p 9999

Basically, all traffic to port 9999 on niobium is forwarded to port 22 on Oxygen -- not terbium.

The cool thing? File transfer is a breeze (via sftp using e.g. filezilla)!

It really is that simple

Web tunnel
to browse the internet via a tunnel it's still easier to do
ssh -C -D 9889 root@terbium.internet.com
and set your browser to use a SOCKS proxy -- point it to localhost, port 9889.

120. Using truecrypt with dropbox


This is not some fancy, automatic solution. This is for people who may not be familiar with truecrypt and how to use it. To most people this will be obvious.

Basically, one way in which you can use truecrypt is to create a file which, when mounted, acts like a device/filesystem. This container is encrypted and the strength of encryption depends on your key or password.

As a solution it also suffers from slow initial syncing.

Setting it up is a piece of cake:
1. Install truecrypt.
Get the correct version e.g. "Standard x64" from http://www.truecrypt.org/downloads.


Install by extracting the file and executing it:
chmod +x truecrypt-7.1a-setup-x64
sudo ./truecrypt-7.1a-setup-x64

2. Start truecrypt and click on Create Volume

Select encrypted file container. The other option is more useful  if you have access to the hardware.

Using a Hidden truecrypt volume means you pay a space penalty, depending on how you distribute the space between the two volumes.

Select location and filename


Pick an algorithm. I don't know much about this, but using a cascade sounds reasonable. I guess there are performance penalties though.

Be aware that the container file will take up all this space -- whether it's empty or not.









 3. Use the container file.
Click on Select File, then mount. You typically need to supply both the container password and your admin password.


This is what the inside of the container file looks like.
And this is what the dropbox folder looks like

19 April 2012

119. Installing sun java/Oracle java in debian

Update 23 March 2013:  the java.com file isn't distributed as a .bin file anymore. Just replace all instances of .bin with .tar.gz and it works just as well.

This basically follows http://sylvestre.ledru.info/blog/sylvestre/2012/02/29/java_package_replacement_of_sun_java6

While a few applications (Grix and Grisu) I was playing with which require java 1.5 or better should work fine with openjdk-7 (1.6.0_24), they didnt. I had to run it using binary packages which I downloaded from java.com and then installed locally.

I'd like a better solution, and here it is:

1. sudo apt-get install java-package

2. Download the java files from java.com, e.g. jre-6u31-linux-x64.bin or jre-7u17-linux-x64.tar.gz
http://java.com/en/download/manual.jsp?locale=en

3. Make-jpkg
Run make-jpkg on the downloaded file, whether it's ending in .bin or .tar.gz:

make-jpkg jre-6u31-linux-x64.bin
Create debian package:
    dh_testdir
    dh_testroot
    dh_installchangelogs
    dh_installdocs
    dh_compress
    dh_fixperms
    dh_installdeb
    dh_shlibdeps
dpkg-shlibdeps: warning: Can't extract name and version from library name `libjvm.so'
[..]
dpkg-shlibdeps: warning: Can't extract name and version from library name `libjvm.so'
[..]
dpkg-shlibdeps: warning: Can't extract name and version from library name `libjli.so'
dpkg-shlibdeps: warning: package could avoid a useless dependency if /tmp/make-jpkg.9XTCruNvKM/install/usr/lib/jvm/j2re1.6-oracle/lib/amd64/native_threads/libhpi.so /tmp/make-jpkg.9XTCruNvKM/install/usr/lib/jvm/j2re1.6-oracle/lib/amd64/libjava.so /tmp/make-jpkg.9XTCruNvKM/install/usr/lib/jvm/j2re1.6-oracle/lib/amd64/libdt_socket.so /tmp/make-jpkg.9XTCruNvKM/install/usr/lib/jvm/j2re1.6-oracle/lib/amd64/libnet.so /tmp/make-jpkg.9XTCruNvKM/install/usr/lib/jvm/j2re1.6-oracle/bin/javaws /tmp/make-jpkg.9XTCruNvKM/install/usr/lib/jvm/j2re1.6-oracle/lib/amd64/libhprof.so were not linked against libnsl.so.1 (they use none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if /tmp/make-jpkg.9XTCruNvKM/install/usr/lib/jvm/j2re1.6-oracle/lib/amd64/libjawt.so was not linked against libmawt.so (it uses none of the library's symbols).
    dh_gencontrol
    dh_md5sums
    dh_builddeb
dpkg-deb: building package `oracle-j2re1.6' in `/tmp/make-jpkg.9XTCruNvKM/oracle-j2re1.6_1.6.0+update31_amd64.deb'.
    copy oracle-j2re1.6_1.6.0+update31_amd64.deb into directory /home/me/Downloads/
The Debian package has been created in the current directory. You can
install the package as root (e.g. dpkg -i oracle-j2re1.6_1.6.0+update31_amd64.deb). 
4. sudo dpkg -i oracle-j2re1.6_1.6.0+update31_amd64.deb 
Selecting previously unselected package oracle-j2re1.6.
(Reading database ... 561777 files and directories currently installed.)
Unpacking oracle-j2re1.6 (from oracle-j2re1.6_1.6.0+update31_amd64.deb) ...
Setting up oracle-j2re1.6 (1.6.0+update31) ...
update-alternatives: using /usr/lib/jvm/j2re1.6-oracle/bin/ControlPanel to provide /usr/bin/ControlPanel (ControlPanel) in auto mode.
update-alternatives: using /usr/lib/jvm/j2re1.6-oracle/lib/amd64/libnpjp2.so to provide /usr/lib/iceweasel/plugins/libjavaplugin.so (iceweasel-javaplugin.so) in auto mode.
update-alternatives: using /usr/lib/jvm/j2re1.6-oracle/lib/amd64/libnpjp2.so to provide /usr/lib/chromium/plugins/libjavaplugin.so (chromium-javaplugin.so) in auto mode.

5a. sudo update-alternatives --config java
There are 5 choices for the alternative java (providing /usr/bin/java).

  Selection    Path                                            Priority   Status------------------------------------------------------------* 0            /usr/lib/jvm/java-6-openjdk-amd64/jre/bin/java   1061      auto mode  1            /usr/bin/gij-4.4                                 1044      manual mode  2            /usr/bin/gij-4.6                                 1046      manual mode  3            /usr/lib/jvm/j2re1.6-oracle/bin/java             314       manual mode  4            /usr/lib/jvm/java-6-openjdk-amd64/jre/bin/java   1061      manual mode  5            /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java   1051      manual modePress enter to keep the current choice[*], or type selection number: 3update-alternatives: using /usr/lib/jvm/j2re1.6-oracle/bin/java to provide /usr/bin/java (java) in manual mode.
5b. 
 sudo update-alternatives --config javaws


There are 2 choices for the alternative javaws (providing /usr/bin/javaws).
  Selection    Path                                              Priority   Status
------------------------------------------------------------
* 0            /usr/lib/jvm/java-6-openjdk-amd64/jre/bin/javaws   1061      auto mode
  1            /usr/lib/jvm/j2re1.6-oracle/bin/javaws             314       manual mode
  2            /usr/lib/jvm/java-6-openjdk-amd64/jre/bin/javaws   1061      manual mode
Press enter to keep the current choice[*], or type selection number: 1
update-alternatives: using /usr/lib/jvm/j2re1.6-oracle/bin/javaws to provide /usr/bin/javaws (javaws) in manual mode.
update-alternatives: warning: skip creation of /usr/share/man/man1/javaws.1.gz because associated file /usr/lib/jvm/j2re1.6-oracle/man/man1/javaws.1.gz (of link group javaws) doesn't exist.



6.  Verification
java -version
java version "1.6.0_31"
Java(TM) SE Runtime Environment (build 1.6.0_31-b04)
Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01, mixed mode)
Go to http://java.com/en/download/installed.jsp


Iceweasel worked fine at this stage.

7. To get Chrome to work I did the following:
sudo updatedb
locate libjavaplugin.so
/usr/lib/chromium/plugins/libjavaplugin.so
/usr/lib/iceweasel/plugins/libjavaplugin.so
/usr/lib/mozilla/plugins/libjavaplugin.so
 ls -lah /usr/lib/chromium/plugins/libjavaplugin.so etc.
/usr/lib/chromium/plugins/libjavaplugin.so -> /etc/alternatives/chromium-javaplugin.so
/usr/lib/iceweasel/plugins/libjavaplugin.so -> /etc/alternatives/iceweasel-javaplugin.so
/usr/lib/mozilla/plugins/libjavaplugin.so -> /etc/alternatives/mozilla-javaplugin.so
ls -lah /etc/alternatives/chromium-javaplugin.so etc.
/etc/alternatives/chromium-javaplugin.so -> /usr/lib/jvm/j2re1.6-oracle/lib/amd64/libnpjp2.so
/etc/alternatives/iceweasel-javaplugin.so -> /usr/lib/jvm/j2re1.6-oracle/lib/amd64/libnpjp2.so
/etc/alternatives/mozilla-javaplugin.so -> /usr/lib/jvm/java-6-openjdk-amd64/jre/lib/amd64/IcedTeaPlugin.so
sudo rm /etc/alternatives/mozilla-javaplugin.so
sudo ln -s /usr/lib/jvm/j2re1.6-oracle/lib/amd64/libnpjp2.so /etc/alternatives/mozilla-javaplugin.so


And you're done.



Links to this post:
https://www.linuxquestions.org/questions/debian-26/which-java-for-wheezy-4175469043/

118. Solution to nwchem: SHMMAX too small

Update: also see this post: http://verahill.blogspot.com.au/2012/10/shmmax-revisited-and-shmall-shmmni.html

When running nwchem using mpirun I've occasionally encountered this error.

Error:
******************* ARMCI INFO ************************
The application attempted to allocate a shared memory segment of 44498944 bytes in size. This might be in addition to segments that were allocated succesfully previously. The current system configuration does not allow enough shared memory to be allocated to the application.

This is most often caused by:
1) system parameter SHMMAX (largest shared memory segment) being too small or
2) insufficient swap space.
Please ask your system administrator to verify if SHMMAX matches the amount of memory needed by your application and the system has sufficient amount of swap space. Most UNIX systems can be easily reconfigured to allow larger shared memory segments,
see http://www.emsl.pnl.gov/docs/global/support.html
In some cases, the problem might be caused by insufficient swap space.
*******************************************************
0:allocate: failed to create shared region : -1
(rank:0 hostname:boron pid:17222):ARMCI DASSERT fail. shmem.c:armci_allocate():1082 cond:0

Diagnosis:
Check the currently defined shmmax:
cat /proc/sys/kernel/shmmax
33554432
Well, 33554432<44498944, so it seems that it's caused by reason 1 above.

Solution:

Edit /etc/sysctl.conf
Add a line saying
kernel.shmmax=44498944
Save and reboot. The exact value is up to you -- I've set my shmmax to 128*1024*1024=134217728, while our production cluster has 6269961216.

Update: to change it on the fly do
sudo sysctl -w kernel.shmmax=6269961216

18 April 2012

117. Sorting out framebuffer in linux (debian)

If you have /dev/fb0 you're fine -- you have a working frame buffer, and that was the case on my laptop and my optiplex 990. However, my homebuilt boxes didn't have any framebuffer devices.

This is how to set up framebuffer:

sudo apt-get install hwinfo
sudo hwinfo --framebuffer

02: None 00.0: 11001 VESA Framebuffer                        
  [Created at bios.464]
  Unique ID: rdCR.fRULN9k9OD4
  Hardware Class: framebuffer
  Model: "Intel(r)Cantiga Graphics Controller"
  Vendor: "Intel Corporation"
  Device: "Intel(r)Cantiga Graphics Controller"
  SubVendor: "Intel(r)Cantiga Graphics Chip Accelerated VGA BIOS"
  SubDevice:
  Revision: "Hardware Version 0.0"
  Memory Size: 127 MB + 960 kB
  Memory Range: 0xd0000000-0xd7feffff (rw)
  Mode 0x0360: 848x480 (+896), 8 bits
  Mode 0x0361: 848x480 (+1728), 16 bits
  Mode 0x0362: 848x480 (+3392), 24 bits
  Mode 0x0305: 1024x768 (+1024), 8 bits
  Mode 0x0317: 1024x768 (+2048), 16 bits
  Mode 0x0318: 1024x768 (+4096), 24 bits  Mode 0x0312: 640x480 (+2560), 24 bits
  Mode 0x0314: 800x600 (+1600), 16 bits
  Mode 0x0315: 800x600 (+3200), 24 bits
  Mode 0x0301: 640x480 (+640), 8 bits
  Mode 0x0303: 800x600 (+832), 8 bits
  Mode 0x0311: 640x480 (+1280), 16 bits
  Config Status: cfg=new, avail=yes, need=no, active=unknown
Pick a mode e.g. 0x0318

Edit your /etc/default/grub

Add vga=0x0318 to the GRUB...DEFAULT line:

GRUB_CMDLINE_LINUX_DEFAULT="quiet nouveau.modeset=0 vga=0x0318"

If there's a nomodeset command, delete it.

Do
sudo update-grub

and reboot. You're done.

There should now be a /dev/fb0 and your resolution when working in a TTY* should be different.

*the terminal you get to by doing alt+f1, alt+f2 etc.

What to do with framebuffers?
Apart from making your ttys look a lot nicer and work at higher resolution, they'll also allow you to use w3m-img for ncurses based browsing with images (w3m+w3m-img; sometimes it's useful), and to take screenshots of the terminal in terminal mode. But mostly, it just makes your terminal experience better through being at higher resolution.


More modes:
02: None 00.0: 11001 VESA Framebuffer                        
  [Created at bios.464]
  Unique ID: rdCR.efxRnBcYXs5
  Hardware Class: framebuffer
  Model: "NVIDIA GF108 Board - 1071v0p1"
  Vendor: "NVIDIA Corporation"
  Device: "GF108 Board - 1071v0p1"
  SubVendor: "NVIDIA"
  SubDevice:
  Revision: "Chip Rev"
  Memory Size: 14 MB
  Memory Range: 0xd7000000-0xd7dfffff (rw)

  Mode 0x0300: 640x400 (+640), 8 bits
  Mode 0x0301: 640x480 (+640), 8 bits
  Mode 0x0303: 800x600 (+800), 8 bits
  Mode 0x0305: 1024x768 (+1024), 8 bits
  Mode 0x0307: 1280x1024 (+1280), 8 bits
  Mode 0x030e: 320x200 (+640), 16 bits
  Mode 0x030f: 320x200 (+1280), 24 bits
  Mode 0x0311: 640x480 (+1280), 16 bits
  Mode 0x0312: 640x480 (+2560), 24 bits
  Mode 0x0314: 800x600 (+1600), 16 bits
  Mode 0x0315: 800x600 (+3200), 24 bits
  Mode 0x0317: 1024x768 (+2048), 16 bits
  Mode 0x0318: 1024x768 (+4096), 24 bits
  Mode 0x031a: 1280x1024 (+2560), 16 bits
  Mode 0x031b: 1280x1024 (+5120), 24 bits
  Mode 0x0330: 320x200 (+320), 8 bits
  Mode 0x0331: 320x400 (+320), 8 bits
  Mode 0x0332: 320x400 (+640), 16 bits
  Mode 0x0333: 320x400 (+1280), 24 bits
  Mode 0x0334: 320x240 (+320), 8 bits
  Mode 0x0335: 320x240 (+640), 16 bits
  Mode 0x0336: 320x240 (+1280), 24 bits
  Mode 0x033d: 640x400 (+1280), 16 bits
  Mode 0x033e: 640x400 (+2560), 24 bits
  Mode 0x0345: 1600x1200 (+1600), 8 bits
  Mode 0x0346: 1600x1200 (+3200), 16 bits
  Mode 0x034a: 1600x1200 (+6400), 24 bits
  Mode 0x0360: 1280x800 (+1280), 8 bits
  Mode 0x0361: 1280x800 (+5120), 24 bits
  Config Status: cfg=new, avail=yes, need=no, active=unknown





Links to this post:
http://pinboard.in/u:greppy

11 April 2012

116. [Solved] Latex bug -- "tcfmgr: config file `tcfmgr.map' (usually in $TEXMFMAIN/texconfig) not found."


Symptom:
When using latexila or texmaker no dvi/ps/pdf is generated. No outright error is report though

When trying to generate the pdf in the terminal using latex e.g.

me@beryllium:~$ latex nature_draft.tex
This is pdfTeX, Version 3.1415926-1.40.10 (TeX Live 2009/Debian)
kpathsea: Running mktexfmt latex.fmt
tcfmgr: config file `tcfmgr.map' (usually in $TEXMFMAIN/texconfig) not found.
I can't find the format file `latex.fmt'!
 kpsewhich -expand-var='$TEXMFMAIN'
gives
/usr/share/texmf/

which is the correct file structure as defined by texmf.cnf

kpsewhich texmf.cnf
/usr/share/texmf/web2c/texmf.cnf

So what the heck is going on? I've written articles, reviews and book in latex and never had a problem until now. Also, it only happens on one computer.

Solution:
I ended up just fooling around. What solved it was doing
sudo texconfig rehash
mktexlsr: Updating /var/lib/texmf/ls-R-TEXMFMAIN...
mktexlsr: Updating /var/lib/texmf/ls-R-TEXLIVE...
mktexlsr: Updating /var/lib/texmf/ls-R...
mktexlsr: Done.
That's all. Really.

Minor note:
you can run texonfig interactively as well. Just make sure to run it using sudo.



Links to this post:
http://tex.stackexchange.com/questions/64894/error-i-cant-find-the-format-file-pdflatex-fmt