Website updated!

My professional webpage, http://www.berkeleychurchill.com, just got a facelift.  Let me know if you like it.  Or more importantly, let me know if you hate it, find a typo, or something does not work right.

Berkeley in kernel land - the conclusion

I should finish up my short-lived series of posts on the little operating system I was working on.  Eventually I discovered that I actually knew everything I needed to know to really build an operating system.  For the longest time it seemed like such an abstract task that I did not know how to do.

But after spending a few weeks working with x86 documentation and actually building a kernel, I realized it was something I could do.  Then the can of worms opened: now that I could build an operating system, what on earth would I build? I could build a macrokernel.  Or a microkernel.  Or an exokernel.  I could map to kernel into the top or bottom half of the memory.  And that is only the very beginning.  The possibilities were limitless, and I could have spent months just planning the operating system I wanted to build.

Because I started with the goal of learning how to build an operating system, I discovered that I had accomplished that goal.  All my system did was read files out of a fat filesystem and execute programs that concurrently printed As, Bs and Cs across my screen.  But my goal was never to build a complete operating system.

So having my goal accomplished, I decided that I did not want to spend the rest of my life building a perfect kernel.  And that's when I stopped.

Fixing a file's "kind" on Mac OS X

Yesterday I had an annoying problem where I was emailed a file and then had to open it using a closed-source program on Mac OS X.  The issue was that the program only wanted to open a file of a certain "kind"; even though all the data was in the file and the file was in-tact the program would refuse to open it. 

Searching on google, there are several utilities that one can install to fix this metadata problem.  However, I was not the administrator of this computer as it was in a lab setting.  Here was a nifty solution.

I had already created several files from this program (Groups and Graphs).  Using a hexeditor I verified that the 'kind' information is definitely stored in the filesystem, and was not based on the contents of the files at all.  If I couldn't change the metadata, I figured I could copy the contents (but not the entire file object) of the file into a file that did have correct metadata.  Here's how this worked.  I had a file, old-good-file, of the correct type that opened.  My new file that wouldn't open was called new-bad-file.

Then in a shell I ran,

cp old-good-file new-good-file
dd if=new-bad-file of=new-good-file

 This way the contents of new-bad-file were moved to new-good-file without changing the metadata of new-good-file.  new-good-file inherited its metadata from old-good-file.  Now new-good-file works perfectly!

Berkeley in Kernel Land (part 2)

I've done a lot of work this week! Who needs classes when you can write operating systems?!?! This week I have,

• Written code to run flat binaries from a filesystem
• Got a few basic syscalls working (write() and exit() so far).  write() only outputs to stdout, which is currently always directly to the screen.
• Made my first live-cds and booted real computers!
• Written a scheduler!!

The scheduler is really the biggest step forward here.  I still have some kinks to work out; my first design worked robustly but lacked the flexibility for me to fully implement sleep, waits, etc.  To improve it I mostly needed to reorganize code -- but after refactoring, it lost some stability.  It seems to work find in the bochs vm and on real hardware but the first sign of trouble was qemu-kvm having problems.  After disabling kvm, qemu did okay for a while.  Now qemu is having issues.  I think I need to go back through my subversion history and figure out when everything went wrong...

Although it's really temping to jump ahead, I want to get this scheduler working cleanly on each of my test platforms before I continue.  I'm really excited to implement VFS and many more syscalls to get newlib working -- but it looks like that will have to hold for a while.

Berkeley in Kernel Land (part 1)

Last Wednesday I decided that I wanted to learn more about operating systems. Sure, I've taken UCSB's operating systems class (and done well in it), but operating systems are so huge and daunting that I felt more was necessary.

So I decided to write one. The goal is to build an operating system that works at some basic level; right now I don't have any further objectives. Ideas have crossed my mind, such as building a system that has a little extra support for stage lighting technology (like DMX-512) among other things. Stage lighting is another hobby of mine.

The OS Dev wiki has been an invaluable source of information! It provides the guidance needed for an experienced programmer with a computer science background to start working on an OS.

So far I've done the following:

• Setup GRUB as a bootloader
• a simple page allocator
• Wrote drivers for vga output, keyboard input, the PIT (programmable timer), the serial port and a dummy ramdisk (which disguises a memory mapped image as a hard drive).
• Wrote filesystem support for FAT-32
• Just last night I built a simple shell that you can use to browse a FAT-32 filesystem

Here are some next steps going forward:

• I need a virtual file system to abstract-away the details of FAT-32
• Code to execute user space programs (for now, just flat binaries -- but in the future I'll use ELF).
  
• A scheduler


• a fancier memory allocator


• Port a C library (perhaps newlib?)


• Port GNU core utils


• More drivers: including ACPI, cdrom, networking, and hard disk


• More filesystems: FAT-16 (easy), ext2/3/4 (harder), others?

Long term, I'm thinking about using modules to replace most drivers and filesystems. Unlike modern linux distributions, my goal is to minimize dynamic module loading and compile as little as possible into the kernel. Hopefully extra needed modules can be loaded from a static list on boot. I'd like to stick to a system that's quite a bit meaner-and-leaner than linux, even if it lacks many capabilities.

Right now GUI is off the radar, but it might be a possibility if I really have time...

That's all for now, but I'm going to try to post updates weekly. By the way, I'm still working on a name for this thing...

Under Attack!!

My roommate had a server running in our living room for a few months now.  I became skeptical about how much it was actually being used, so I decided to check out the /var/log/auth.log file and count logins.  After filtering out all the cron jobs, I immediately noticed a few suspicious lines:

    May  2 18:56:29 Mc-server sshd[27075]: Invalid user angela from 58.30.236.115
    May  2 18:56:29 Mc-server sshd[27075]: pam_unix(sshd:auth): check pass; user unknown
    May  2 18:56:29 Mc-server sshd[27075]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=58.30.236.115
    May  2 18:56:30 Mc-server sshd[27075]: Failed password for invalid user angela from 58.30.236.115 port 58943 ssh2

Ok, um, I don't know of any Angelas who live in my house.  Sketchy.  I ran GeoIP on the IP address, and guess what?  It's Chinese!  Unless my roommate has more international connections than I suspected, this is quite unusual.


    grep /var/log/auth.log 58.30.236.115 | wc

       3807   52210  440164


Huh.  Now we're talking about a small password cracking attempt.  It turns out that we actually were attacked four times within a 24-hour period.  Our logs only date back two days (they were cluttered with CRON entries), so we really don't know what happened before that.


Attack 1
06:16:01 to 06:34:42
From 119.161.145.206
20 attacks against 'root'
212 attacks against other accounts

Attack 2
09:09:26 to 09:10:25
From 125.208.5.78
11 attacks against 'root'
4 attacks against other accounts

Attack 3
17:10:09 to 18:56:30
From 58.30.236.115
576 attacks against 'root'
900 attacks against other accounts

Attack 4
22:49:55
from 125.88.105.43
A single attack without a valid authentication string.  Perhaps trying to exploit some old vulnerability?


SUMMARY:
* A total of 1703 attempted logins
* 607 attempts made against 'root'
* 166 attempts made against 'admin'
* The other 930 attacks were made against 531 usernames that appear to have been chosen from English dictionaries
* They never correctly guessed an active username on the system (other than root and some common system names).

Just for good measure, in the end we brought the system offline and checked for any malware/rootkits.  To no surprise, nothing suspicious was found.

So why did this happen to us?  When the SSH server was setup it was put on the default port 22.  This is just asking for trouble.  Never ever do this.  While we do not believe the system was ever compromised its just an invitation for problems.

What else do you learn from this?  (#1) By default there is no notification system to warn you of suspicious activity.  A cron job would be an easy fix.  (#2) Never give default system accounts (especially "root") SSH access.  This is the biggest target.  (#3) Don't use port 22, or other common service ports, if practical.



Of course, there are far more things you can do to protect your SSH servers. These suggestions really only graze the surface and should get you started on a better configuration for your servers.

More xfce

1.  My menu generating script can be found at https://sourceforge.net/projects/xfce-menu-maker/.  While there's only one svn commit, it's pretty stable (I've worked on it for a while).  Let me know if you find any bugs.

2.  Did you realize you can drag windows from one workspace to another INSIDE the workspace switcher?  That's right, just click on that little firefox icon and drag it to your screen.

3.  I am in love with xfce's extensive options for window transparency and focus.  It can provide a very different and more productive desktop interface.

grep and columns in color!

I really wanted my dictionary searches to output in columns, but without loosing grep's color.  The problem is that if you use the --color=always option in grep it distorts the length of the strings in the pipeline because of the escape sequences used for coloring in the terminal.  Then when you pipe the results into column the columns do not look right.

Never fear.  awk is here.  I wrote this tiny script to append spaces to each line so they all had at least length 35.

Here's normalize.awk

#!/usr/bin/gawk -f

{
    str = $1
    difference = 35 - length($1);
    while(difference > 0)
    {
        str = str " "
        difference--
    }
    print str
}

and here's the finished command:

grep --color=always st$ dict | ./normalize.awk | column

This command found all the words in the dictionary ending in "st" and displays them in columns with the "st" portion highlighted.

xfce tricks, tips and cool features

I love xfce.  In all fairness, I haven't used gnome for over a year, so I cannot really make a fair comparison to the newer versions.  But I would like to say some reasons I do love xfce...

I would just like to highlight some recent tricks and tips in xfce that I think are really cool, and worth knowing about:

1. You can use the middle mouse button to bring any window into focus, regardless of what workspace its on.
2. If there are two or more windows on top of each other, say maximized, then middle-clicking the title bar cycles them into focus.
3. Dual monitor configuration works pretty well once you have X configured properly.  Sometimes you need to restart x for everything to work right the first time.  For example, you can move panels between monitors using the panels menu.
4. If you want to duplicate a panel, so it shows up on two monitors, there's no  GUI way to do this.  But if you edit the config file ~/.config/xfce4/panel/panels.xml, you can simply copy one of the <panel>....</panel> entries and paste it somewhere else.  You should them change the screen-position or the monitor properties of one of them to move it (although strictly-speaking, this is probably optional).  Then 'pkill xfce4-panel' and restart it (best to hit Alt-F2, type "xfce4-panel" and hit enter).  If you changed the screen-position/monitor settings for your new panel you should see it immediately.  Otherwise you'll need to go to the xfce4->settings->Panel window, and you'll see there's a duplicate panel in the list that you can move around.
5. You can right click on the desktop to bring up the applications menu
6. Soon I'll be publishing a python script that takes a folder of scripts and turns it into a launcher with a menu on one of your panels... stay tuned :)

And some things I've known about for a while that I use all the time:

1. You can configure windows to be partially transparent when you move them, so you can see all the windows behind as you're rearranging things
2. You can set a window to 'always on top' or 'always on bottom'.  This often helps when you're forced to overlap windows, and you want to read from one while editing the other.  Typically I put the smaller window "on top" in such a way I can still see everything I need on the bottom one.

Crossplatform Todo lists!

I've had this problem for a long time.  I want a simple todo-list solution that satisfies the following requirements:

1. Shows up on my computers and my android phone
• The Android app must be convenient
• Either there needs to be a really convenient desktop application, or it has to integrate into a web site I already use, such as the iGoogle homepage
2. I shouldn't have to develop it myself
3. Streamlined synchronization.  I shouldn't have to manually sync it all the time.
4. Features preserved in synchronization.  For example, if on my phone a todo item
5. Preferably everything goes over SSL
6. Automatically sorts tasks by priority

I tried a few things. 

• I started with mnote on my phone.  This didn't get along with my desktop very easily.
• I thought about just keeping a svn repository with a flat text file and adding a hook to do sorting by the first item on the list.  Unfortunately, this is a lot of work and doesn't play nicely with the phone
• I thought a lot about setting up my own server with an Apache Thrift service.  Too much work.
• I looked at several web services, but none of them seemed to really do the trick.
• I tried astrid tasks, but the syncing wasn't very pretty with google tasks or producteev.  Producteev also seemed way overkill.
• Then I tried Remember The Milk... WIN!

www.rememberthemilk.com.  This does all the above and more!  The downside is that it looks like I'll need to fork over some money after my trial period is over if I want to keep the android app.  It looks like it will be $25/year.  Not excited about that... but it's not too much to pay for a good service.

I'm not totally sure that everything is SSL'ed.  I know that when I work directly with the webapplication it is, but I don't think it is from the iGoogle page or the android app, but I have yet to actually confirm or deny this.