cd is fancier than you think!

I was doing something-or-other today, when I had the urge to run "help cd".  What I found changed my life.  I will never cd the same way again.  It was so life-changing I'm taking a short break from my PLDI paper, which is due in about a week.

Apparently, there's this environment variable $CDPATH which has a list of places that cd searches for the name of the folder you specified.  For example, if you have folders like

cs242
cs300
cs1337

inside your ~/school folder, and you set CDPATH=~/school, then typing "cd cs242" will take you to ~/school/cs242, no matter what your current working directory is.

But now, I have this problem: what if there are some folders in ~ that I visit often, and others that I don't want to accidentally go to (or have short names, like ~/tmp).  Symbolic links to the rescue!

Create a folder like ~/.cddir.  Inside ~/.cddir you can create symbolic links to any folder you would like to go to.  Then add ~/.cddir to your path, and you're set!

There's one more bit of awkwardness.  If you have a symbolic link like

~/.cddir/school -> ~/school

and then you do "cd school", you'll end up in ~/.cddir/school, which isn't what you really want, because if you do "cd ..", you would prefer to be in ~, rather than ~/.cddir.  "help cd" comes back to the rescue and advises to use "cd -P" instead.  This fixes the issue.  The easy thing to do is make this an alias in your .bashrc.

To recap: go into your .bashrc and add the following:

alias cd='cd -P'
export CDPATH='~/.cddir'

Then:

cd ~
mkdir .cddir
cd .cddir

and add symbolic links like

ln -s ~/school school.

Then you would be able to use 'cd school' to get to your school folder from any working directory!

Website updated!

My professional webpage, http://www.berkeleychurchill.com, just got a facelift.  Let me know if you like it.  Or more importantly, let me know if you hate it, find a typo, or something does not work right.

Berkeley in kernel land - the conclusion

I should finish up my short-lived series of posts on the little operating system I was working on.  Eventually I discovered that I actually knew everything I needed to know to really build an operating system.  For the longest time it seemed like such an abstract task that I did not know how to do.

But after spending a few weeks working with x86 documentation and actually building a kernel, I realized it was something I could do.  Then the can of worms opened: now that I could build an operating system, what on earth would I build? I could build a macrokernel.  Or a microkernel.  Or an exokernel.  I could map to kernel into the top or bottom half of the memory.  And that is only the very beginning.  The possibilities were limitless, and I could have spent months just planning the operating system I wanted to build.

Because I started with the goal of learning how to build an operating system, I discovered that I had accomplished that goal.  All my system did was read files out of a fat filesystem and execute programs that concurrently printed As, Bs and Cs across my screen.  But my goal was never to build a complete operating system.

So having my goal accomplished, I decided that I did not want to spend the rest of my life building a perfect kernel.  And that's when I stopped.

Fixing a file's "kind" on Mac OS X

Yesterday I had an annoying problem where I was emailed a file and then had to open it using a closed-source program on Mac OS X.  The issue was that the program only wanted to open a file of a certain "kind"; even though all the data was in the file and the file was in-tact the program would refuse to open it. 

Searching on google, there are several utilities that one can install to fix this metadata problem.  However, I was not the administrator of this computer as it was in a lab setting.  Here was a nifty solution.

I had already created several files from this program (Groups and Graphs).  Using a hexeditor I verified that the 'kind' information is definitely stored in the filesystem, and was not based on the contents of the files at all.  If I couldn't change the metadata, I figured I could copy the contents (but not the entire file object) of the file into a file that did have correct metadata.  Here's how this worked.  I had a file, old-good-file, of the correct type that opened.  My new file that wouldn't open was called new-bad-file.

Then in a shell I ran,

cp old-good-file new-good-file
dd if=new-bad-file of=new-good-file

 This way the contents of new-bad-file were moved to new-good-file without changing the metadata of new-good-file.  new-good-file inherited its metadata from old-good-file.  Now new-good-file works perfectly!

Berkeley in Kernel Land (part 2)

I've done a lot of work this week! Who needs classes when you can write operating systems?!?! This week I have,

• Written code to run flat binaries from a filesystem
• Got a few basic syscalls working (write() and exit() so far).  write() only outputs to stdout, which is currently always directly to the screen.
• Made my first live-cds and booted real computers!
• Written a scheduler!!

The scheduler is really the biggest step forward here.  I still have some kinks to work out; my first design worked robustly but lacked the flexibility for me to fully implement sleep, waits, etc.  To improve it I mostly needed to reorganize code -- but after refactoring, it lost some stability.  It seems to work find in the bochs vm and on real hardware but the first sign of trouble was qemu-kvm having problems.  After disabling kvm, qemu did okay for a while.  Now qemu is having issues.  I think I need to go back through my subversion history and figure out when everything went wrong...

Although it's really temping to jump ahead, I want to get this scheduler working cleanly on each of my test platforms before I continue.  I'm really excited to implement VFS and many more syscalls to get newlib working -- but it looks like that will have to hold for a while.

Berkeley in Kernel Land (part 1)

Last Wednesday I decided that I wanted to learn more about operating systems. Sure, I've taken UCSB's operating systems class (and done well in it), but operating systems are so huge and daunting that I felt more was necessary.

So I decided to write one. The goal is to build an operating system that works at some basic level; right now I don't have any further objectives. Ideas have crossed my mind, such as building a system that has a little extra support for stage lighting technology (like DMX-512) among other things. Stage lighting is another hobby of mine.

The OS Dev wiki has been an invaluable source of information! It provides the guidance needed for an experienced programmer with a computer science background to start working on an OS.

So far I've done the following:

• Setup GRUB as a bootloader
• a simple page allocator
• Wrote drivers for vga output, keyboard input, the PIT (programmable timer), the serial port and a dummy ramdisk (which disguises a memory mapped image as a hard drive).
• Wrote filesystem support for FAT-32
• Just last night I built a simple shell that you can use to browse a FAT-32 filesystem

Here are some next steps going forward:

• I need a virtual file system to abstract-away the details of FAT-32
• Code to execute user space programs (for now, just flat binaries -- but in the future I'll use ELF).
  
• A scheduler


• a fancier memory allocator


• Port a C library (perhaps newlib?)


• Port GNU core utils


• More drivers: including ACPI, cdrom, networking, and hard disk


• More filesystems: FAT-16 (easy), ext2/3/4 (harder), others?

Long term, I'm thinking about using modules to replace most drivers and filesystems. Unlike modern linux distributions, my goal is to minimize dynamic module loading and compile as little as possible into the kernel. Hopefully extra needed modules can be loaded from a static list on boot. I'd like to stick to a system that's quite a bit meaner-and-leaner than linux, even if it lacks many capabilities.

Right now GUI is off the radar, but it might be a possibility if I really have time...

That's all for now, but I'm going to try to post updates weekly. By the way, I'm still working on a name for this thing...

Under Attack!!

My roommate had a server running in our living room for a few months now.  I became skeptical about how much it was actually being used, so I decided to check out the /var/log/auth.log file and count logins.  After filtering out all the cron jobs, I immediately noticed a few suspicious lines:

    May  2 18:56:29 Mc-server sshd[27075]: Invalid user angela from 58.30.236.115
    May  2 18:56:29 Mc-server sshd[27075]: pam_unix(sshd:auth): check pass; user unknown
    May  2 18:56:29 Mc-server sshd[27075]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=58.30.236.115
    May  2 18:56:30 Mc-server sshd[27075]: Failed password for invalid user angela from 58.30.236.115 port 58943 ssh2

Ok, um, I don't know of any Angelas who live in my house.  Sketchy.  I ran GeoIP on the IP address, and guess what?  It's Chinese!  Unless my roommate has more international connections than I suspected, this is quite unusual.


    grep /var/log/auth.log 58.30.236.115 | wc

       3807   52210  440164


Huh.  Now we're talking about a small password cracking attempt.  It turns out that we actually were attacked four times within a 24-hour period.  Our logs only date back two days (they were cluttered with CRON entries), so we really don't know what happened before that.


Attack 1
06:16:01 to 06:34:42
From 119.161.145.206
20 attacks against 'root'
212 attacks against other accounts

Attack 2
09:09:26 to 09:10:25
From 125.208.5.78
11 attacks against 'root'
4 attacks against other accounts

Attack 3
17:10:09 to 18:56:30
From 58.30.236.115
576 attacks against 'root'
900 attacks against other accounts

Attack 4
22:49:55
from 125.88.105.43
A single attack without a valid authentication string.  Perhaps trying to exploit some old vulnerability?


SUMMARY:
* A total of 1703 attempted logins
* 607 attempts made against 'root'
* 166 attempts made against 'admin'
* The other 930 attacks were made against 531 usernames that appear to have been chosen from English dictionaries
* They never correctly guessed an active username on the system (other than root and some common system names).

Just for good measure, in the end we brought the system offline and checked for any malware/rootkits.  To no surprise, nothing suspicious was found.

So why did this happen to us?  When the SSH server was setup it was put on the default port 22.  This is just asking for trouble.  Never ever do this.  While we do not believe the system was ever compromised its just an invitation for problems.

What else do you learn from this?  (#1) By default there is no notification system to warn you of suspicious activity.  A cron job would be an easy fix.  (#2) Never give default system accounts (especially "root") SSH access.  This is the biggest target.  (#3) Don't use port 22, or other common service ports, if practical.



Of course, there are far more things you can do to protect your SSH servers. These suggestions really only graze the surface and should get you started on a better configuration for your servers.

More xfce

1.  My menu generating script can be found at https://sourceforge.net/projects/xfce-menu-maker/.  While there's only one svn commit, it's pretty stable (I've worked on it for a while).  Let me know if you find any bugs.

2.  Did you realize you can drag windows from one workspace to another INSIDE the workspace switcher?  That's right, just click on that little firefox icon and drag it to your screen.

3.  I am in love with xfce's extensive options for window transparency and focus.  It can provide a very different and more productive desktop interface.

grep and columns in color!

I really wanted my dictionary searches to output in columns, but without loosing grep's color.  The problem is that if you use the --color=always option in grep it distorts the length of the strings in the pipeline because of the escape sequences used for coloring in the terminal.  Then when you pipe the results into column the columns do not look right.

Never fear.  awk is here.  I wrote this tiny script to append spaces to each line so they all had at least length 35.

Here's normalize.awk

#!/usr/bin/gawk -f

{
    str = $1
    difference = 35 - length($1);
    while(difference > 0)
    {
        str = str " "
        difference--
    }
    print str
}

and here's the finished command:

grep --color=always st$ dict | ./normalize.awk | column

This command found all the words in the dictionary ending in "st" and displays them in columns with the "st" portion highlighted.

xfce tricks, tips and cool features

I love xfce.  In all fairness, I haven't used gnome for over a year, so I cannot really make a fair comparison to the newer versions.  But I would like to say some reasons I do love xfce...

I would just like to highlight some recent tricks and tips in xfce that I think are really cool, and worth knowing about:

1. You can use the middle mouse button to bring any window into focus, regardless of what workspace its on.
2. If there are two or more windows on top of each other, say maximized, then middle-clicking the title bar cycles them into focus.
3. Dual monitor configuration works pretty well once you have X configured properly.  Sometimes you need to restart x for everything to work right the first time.  For example, you can move panels between monitors using the panels menu.
4. If you want to duplicate a panel, so it shows up on two monitors, there's no  GUI way to do this.  But if you edit the config file ~/.config/xfce4/panel/panels.xml, you can simply copy one of the <panel>....</panel> entries and paste it somewhere else.  You should them change the screen-position or the monitor properties of one of them to move it (although strictly-speaking, this is probably optional).  Then 'pkill xfce4-panel' and restart it (best to hit Alt-F2, type "xfce4-panel" and hit enter).  If you changed the screen-position/monitor settings for your new panel you should see it immediately.  Otherwise you'll need to go to the xfce4->settings->Panel window, and you'll see there's a duplicate panel in the list that you can move around.
5. You can right click on the desktop to bring up the applications menu
6. Soon I'll be publishing a python script that takes a folder of scripts and turns it into a launcher with a menu on one of your panels... stay tuned :)

And some things I've known about for a while that I use all the time:

1. You can configure windows to be partially transparent when you move them, so you can see all the windows behind as you're rearranging things
2. You can set a window to 'always on top' or 'always on bottom'.  This often helps when you're forced to overlap windows, and you want to read from one while editing the other.  Typically I put the smaller window "on top" in such a way I can still see everything I need on the bottom one.

Crossplatform Todo lists!

I've had this problem for a long time.  I want a simple todo-list solution that satisfies the following requirements:

1. Shows up on my computers and my android phone
• The Android app must be convenient
• Either there needs to be a really convenient desktop application, or it has to integrate into a web site I already use, such as the iGoogle homepage
2. I shouldn't have to develop it myself
3. Streamlined synchronization.  I shouldn't have to manually sync it all the time.
4. Features preserved in synchronization.  For example, if on my phone a todo item
5. Preferably everything goes over SSL
6. Automatically sorts tasks by priority

I tried a few things. 

• I started with mnote on my phone.  This didn't get along with my desktop very easily.
• I thought about just keeping a svn repository with a flat text file and adding a hook to do sorting by the first item on the list.  Unfortunately, this is a lot of work and doesn't play nicely with the phone
• I thought a lot about setting up my own server with an Apache Thrift service.  Too much work.
• I looked at several web services, but none of them seemed to really do the trick.
• I tried astrid tasks, but the syncing wasn't very pretty with google tasks or producteev.  Producteev also seemed way overkill.
• Then I tried Remember The Milk... WIN!

www.rememberthemilk.com.  This does all the above and more!  The downside is that it looks like I'll need to fork over some money after my trial period is over if I want to keep the android app.  It looks like it will be $25/year.  Not excited about that... but it's not too much to pay for a good service.

I'm not totally sure that everything is SSL'ed.  I know that when I work directly with the webapplication it is, but I don't think it is from the iGoogle page or the android app, but I have yet to actually confirm or deny this.

How to run vim commands for certain file types

For the longest time I've created .tex files in vim and typeset them with pdflatex.  The problem is that every time I need to type

:set makeprg=pdflatex\ %

This was frustrating because I use tex all the time.  Alas, the solution is to place the following in your ~/.vimrc:

autocmd FileType tex setlocal makeprg=pdflatex\ %

LaTeXCalc 1.0 is out!

Ever been working on a homework set or a paper in LaTeX where you constantly resort to a calculator to do boring computations?  LaTeXCalc is your solution.  It is a "LaTeX Calculator" that allows you to in-line computations with TeX.  See the homepage at http://latexcalc.sourceforge.net.

Currently available on UNIX-like platforms (aka almost everything except Windows).  A Windows port will be available "soon".

Thoughts on Wikipedia

My close friend Jason recently got back from studying abroad in Ghana*.  Yesterday he was telling me how the classes in Ghana weren't well put together and often just involved professors reading from scripts.  Upon asking where the scripts were from he promptly replied "Wikipedia".

At first this is a little disappointing.  We all hope that education around the would could be better.  On the other hand, this means that the whole world can influence education in Ghana and other third world countries!

This is one of many good reasons to edit wikipedia.  As odd as it may sound, it has the power to educate and transform the world.  Undoubtedly you can contribute something too.  My biggest request is that you take care when you edit.  Don't just write.  Write quality material and follow Wikipedia's guidelines.  Each small contribution has a very real effect on every single person who visits the page.  Wikipedia has helped you, now return the favor!



* On a side note, Jason's compelling blog of his Africa journeys may be found at http://jasonghana.blogspot.com/.

Hacking XScreensaver to take photos of intruders

Once upon a time, I returned to my computer to see the message "11 Failed Login Attempts" when logging in.  Needless to say, this was unnerving.  To catch the culprit I successfully recompiled XScreensaver to take pictures of anyone getting my password wrong.  Here's how I did it.

Disclaimer: This uses an extremely cheap hack. In fact, I wouldn't be surprised if it introduces subtle security vulnerabilities, but I don't know of any.  (EDIT: it turns out that even though the system() command is given an absolute path, the shell environment can be modified to abuse it.  Since xscreensaver is a setuid program, this could lead to elevation of privilege attacks.)  Use at your own risk.

When I originally did this I used Arch Linux, but since then I've ported it to Ubuntu and it should work on any *nix system with tweaks.  This won't work for Windows, unless you can get xscreensaver and mplayer working in cygwin (good luck!).

First get your webcam setup and /dev/video0 working. You can test this with a program like cheese.  Install mplayer. Then

$mplayer tv:// -tv driver=v4l2:device=/dev/vi

deo0:width=800:height=600:outfmt=rgb24 -frames 4 -vo jpeg:outdir="/data/berkeley/failed-login-pics/`date +"%Y-%B-%d"`/`date +"%H.%M.%S"`" > /dev/null 2> /data/berkeley/failed-login-pics/mplayer-error-log

will take four pictures using /dev/video0 and v4l2 drivers and save it to /data/berkeley/failed-login-pics/date/time.  Obviously you'll want to change the paths for your system.  You may need to adjust other settings as well.  Get this working before continuing.  Place this command in a script of your choice somewhere in your system.

Here's how to make xscreensaver run this command when someone guesses your password wrong:

Download the xscreensaver source and modify xscreensaver-version/driver/lock.c as follows (I've done so much other stuff to this file I can't really provide a patch or diff):

1. Insert this line at the top, under the other standard includes

#include <stdlib.h>

2. Add a function int takePicture() as follows near the top somewhere. Use the command that worked for your computer above in the system("...") call! Be sure to escape every " with \. Don't ignore the mkdir command. Also note that there are actually only two lines in the body of the function.

int takePicture() {
     system("/path/to/script");
}

3. Somewhere around line 2040 you'll see code that looks like:

if (si->unlock_state == ul_fail && /* failed with caps lock on */

si->pw_data && si->pw_data->caps_p)
     s = "Authentication failed! (caps lock)";
else if (si->unlock_state == ul_fail) /* failed without caps lock */
     s = "Authentication failed!";

change it to this: (and feel free to customize the messages)

if (si->unlock_state == ul_fail && /* failed with caps lock on */
si->pw_data && si->pw_data->caps_p)
{
     takePicture();
     s = "Authentication FAIL!\n (caps lock is on)";
}
else if (si->unlock_state == ul_fail) /* failed without caps lock */
{
     takePicture();
     s = "Authentication FAIL!\n (this will be reported)";
}

4. While you're at it, I highly recommend changing the xscreensaver pictures/icons and replacing them with whatever you want. I used arch linux icons. You'll find them in one of the source directories.

5. If you're crafty you can also modify some of the other visual elements in the login screen. For example, at somewhere around line 250 you can modify the strings that contain "xscreensaver blahblahblah" and the hostname to whatever you want. Be careful! If you write it wrong or don't use strdup you could create segmentation faults when xscreensaver frees up the window resources causing xscreensaver to crash. If you do this then pressing the ESC button will bypass the login window.  Test your modifications throughly. 


Install can be a bit tricky. First remove xscreensaver and gnome screensaver from any package managers, eg.

$sudo apt-get remove xscreensaver
$sudo apt-get remove gnome-screensaver

OR

$sudo pacman -R xscreensaver
$sudo pacman -R gnome-screensaver

Then execute

$./configure --prefix=/usr
$make
$sudo make install

To test it out, do the following

$pkill xscreensaver   (or $killall xscreensaver)

$xscreensaver &
$xscreensaver-command -lock

When you test, make sure that

1) STDOUT and STDERR of mplayer and mkdir are suppressed or logged (you'll know if it's not)
2) You can't bypass the login screen by pressing ESC, waiting, overflowing some buffer, or otherwise causing a segfault.

For a GUI configuration editor, try

$xscreensaver-demo

Finally, make sure that xscreensaver is run at startup with your desktop manager so it actually turns on.

Have fun.

Adding a new server to the MINIX 3.1.6 kernel.

For a class project I was required to add a new service to the MINIX kernel.  In MINIX, all the essential parts of the operating system are implemented as services.  For example, the file system is a service, the process manager is a service and so forth.

To my dismay, no documentation on adding a service exists (or is hard to find), except in my project's description.  Unfortunately, that too was outdated.  After much experimentation, here's what I discovered needs to be done to add a service:

(1)  Copy an existing service to create foundations.  Most services need the same basic structure: a dispatch loop that checks for messages and executes system calls.  The services have source code in /usr/src/services.  Simple servers to copy include ds (data store server) and is (information server).  For example:

$cp -r /usr/src/servers/ds /usr/src/servers/foo

Here, "foo" is the abreviated name for the new service.

• You should modify the Makefile in your /usr/src/servers/foo folder to update the name of the binary.  You should only have to change the first or second line.
• It would be a good idea to appropriately modify the Makefile in /usr/src/servers at this point as well.
• Once you get everything working, you should remove all the code from the other service that's not relevant to you.  When I created my server from the ds server I ended up deleting store.c, store.h and removed calls from main.c to functions in those files.

(2)  Add "foo" to the boot image.  Since MINIX uses a microkernel architecture there are several places where boot image information needs to be updated.  In general, the order that services are listed in tables are important as it determines the order they startup.

• /usr/src/tools/Makefile contains the Makefile that compiles to boot image.  You must add the path to your binary in the 'programs' variable.
• /usr/src/include/minix/com.h lists constants for all of the boot image processes.  You'll see entries like "#define DS_PROC_NR ..." for each process.  You'll need to create your own constant for your process and update the constants on the other ones so the order is consistent.
• /usr/src/kernel/table.c has a table of the boot image processes that the kernel uses.  This needs to be updated and is fairly self explanatory.  You will want to copy values from another process for the fields.  You could change the values if you know what you're doing, but you don't really need to.
• /usr/src/servers/rs/table.c has THREE tables that you'll need to fill out.  Again, I would recommend copying entries from other processes.
• /usr/src/servers/rs/manager.c has a list of if statements around line 1645 that needs to be updated with information for your process.

(3) That should be it!  You should recompile the kernel using 'make fresh install' from the /usr/src/tools directory.  It should be possible to compile it using a command like 'make install' or 'make hdboot' instead, but it's sometimes hard to know.  You should reboot to make sure your service starts up, and then you can start hacking!