A mind less ordinary » Personal

Useful git configuration fragments

Dave — Wed, 18 Aug 2010 13:17:48 +0000

Time for a quick micro-post, I think. I feel like sharing two useful fragments of my git configuration.

First up, aliases. I have a few useful shortcuts defined, as well as some nice ways of displaying a repository’s history. I’ve aliased many of the common commands to two-letter versions which evoke the similar Subversion command aliases:

[alias]

  st = status

  di = diff

  co = checkout

  ci = commit

  br = branch

  sta = stash

  graph = log --decorate --oneline --graph --all --date-order

  lg = log --graph --pretty=format:'%C(yellow)%h%Creset -%C(yellow)%d%Creset %s %C(green)(%cr)%Creset %Cred<%an>%Creset' --abbrev-commit --date=relative --date-order

The last two aliases above provide a nice graphical view of history in the console. git graph displays all history of all branches in date order rather than topological order. This means that commits on different branches may be interleaved, rather than grouping commits on different branches. git lg displays a little more information (including author and relative timestamps), and allows me to pick which branches I want to see. The examples below are all taken from the Jerity repository.

The second tip is repository URL rewriting. This tackles the annoying problem of having submodules in projects that you want to be able to commit to, while making sure other people only see/use the read-only URLs. Git will very helpfully rewrite URLs from one form to another, so by having:

[url "ssh://git@github.com/dingram/"]

  pushInsteadOf = git://github.com/dingram/

in my configuration, I can git clone git://github.com/dingram/repository and when I push, it will automatically push to git@github.com:dingram/repository instead. This can be verified using git remote -v. The only thing to beware of is the format: the replacement URL goes in the section header, and the pattern to be replaced appears in the section itself.

Recovering from pvmove failure

Dave — Thu, 07 Jan 2010 23:25:29 +0000

Note: This is a fairly rambling explanation of recent events. I assume that you have at least a passing knowledge of LVM and its terminology. This was written to prove that it is possible to recover from pvmove failing (in certain cases) due to the way it performs its operations and backs up metadata.

The background

I recently received a new hard drive, and so I excitedly partitioned and formatted it, extended my LVM VGs and merrily moved my LVs completely across to the new drive using pvmove. So far, so wonderful — I was now ready to rebuild my other two drives so that only critical things were covered by RAID and everything else was just in a large VG that spanned multiple drives. I now had three drives, all Samsung SpinPoint:

500GB (sdb)
1TB (sda)
1.5TB (sdc, and brand-new)

The partition setup I wanted:

48MiB RAID-1 across all drives, for /boot (already set up across sda and sdb)
4.8GiB RAID-1 across all drives, for /(already set up across sda and sdb)
50GiB RAID-1 across all drives, for /usr, /opt, /var
50GiB RAID-5 across all drives, for /home (giving 100GiB usable space)
The rest of each drive divided into 50/100GiB partitions, to be spread among my large data VG, containing films, music, backups, and other large data.

The disaster

I moved all of the existing LVs to the new drive, which had plenty of space. Next, I sorted out the partitions for the two older drives, and initialised the RAIDs for my sys and safe VGs:

# set up RAID-1 across /dev/sda3, /dev/sdb3 and (later) /dev/sdc3
mdadm -C /dev/md2 -l1 -n3 /dev/sda3 /dev/sdb3 missing
# initialise the RAID for LVM
pvcreate /dev/md2
# set up RAID-5 across /dev/sda5, /dev/sdb5 and (later) /dev/sdb5
mdadm -C /dev/md3 -l5 -n3 /dev/sda5 /dev/sdb5 missing
pvcreate /dev/md3

I then expanded the VGs and started to copy data across. I began with the safe VG that contained /home, as I wanted to be sure it was indeed safe. I have all of the photos I’ve ever taken, my email backups and archives, and various private keys and other important data, as well as things that could be replaced due to duplication elsewhere (including working copies of most of my projects). Being LVM, I could move the LVs around without needing to unmount them or reboot into a LiveCD:

# extend the volume groups
vgextend sys /dev/md2
vgextend safe /dev/md3
# move all data from /dev/sdc2 to other volumes in the group
pvmove /dev/sdc2
# ... disaster struck before the second pvmove
pvmove /dev/sdc1

Then, disaster struck: pvmove spat a set of I/O errors at me… but kept running! I wasn’t paying attention (as moving 40GiB of data takes quite a while) and so I had just left it to work away.

pvmove: dev_open(/dev/sda) called while suspended
pvmove: dev_open(/dev/sda1) called while suspended
pvmove: dev_open(/dev/sda2) called while suspended
pvmove: dev_open(/dev/sda3) called while suspended
pvmove: dev_open(/dev/sda5) called while suspended
pvmove: dev_open(/dev/md2) called while suspended
pvmove: dev_open(/dev/md3) called while suspended

It was only after it had completed and I started getting I/O and “permission denied” errors while trying to access my home directory that I first realised anything had gone wrong. After some frantic searching on the Internet, it seemed that I had to wave goodbye to all of my personal data, including those bits that weren’t backed up (though I’ve learned my lesson about backups now!).

I was determined not to give in to data loss, and so I gave myself a few recovery options:

Create an image of the partition where the data used to reside.
Try to recover pictures (the most critical thing for me) via foremost, which is a non-destructive operation.
Try to rebuild the file system using reiserfsck --rebuild-tree.

During my investigations and poking around, I discovered that there was another option: I could try to rebuild the VG from an old copy of its metadata.

Introduction to recovery

It turns out that many LVM utilities store a backup of the VG’s metadata before and after performing various operations, so as long as your /etc/lvm directory is fine, you have a fighting chance. (Actually, it is theoretically possible to retrieve old versions of the metadata from the LVM partition itself, but this gets much more complex and I won’t deal with it here.)

The reason for this is that pvmove works by creating a temporary target LV, mirroring data from the existing LV to the temporary target (making checkpoints along the way), and only removing the original once the mirroring has completed. This operation can then be interrupted and resumed at any time without loss of data. This also has the advantage that a full copy of the data on the original LV is still available; only the metadata explaining where it begins and ends has changed.

To access the old versions of this metadata, investigate the contents of your /etc/lvm/archive directory as root. You should find one or more files of the form volgroup_nnnnn.vg, where volgroup is the name of the relevant VG and nnnnn is a number. These files contain a backup of the volume group’s metadata, such as which partitions it resides on and where it’s located. The contents may look something like this:

# Generated by LVM2 version 2.02.56(1) (2009-11-24): Mon Jan 4 17:09:24 2010

contents = "Text Format Volume Group"
version = 1

description = "Created *before* executing 'vgextend safe /dev/md/3'"

creation_host = "hostname" # Linux hostname 2.6.31-gentoo-r3 #3 SMP Mon Nov 23 11:31:59 GMT 2009 i686
creation_time = 1262624964 # Mon Jan 4 17:09:24 2010

safe {
id = "zbSAI8-ExUy-PxJw-SHX7-y9PL-AL2y-kiTk8g"
seqno = 10
status = ["RESIZEABLE", "READ", "WRITE"]
flags = []
extent_size = 8192 # 4 Megabytes
max_lv = 0
max_pv = 0

physical_volumes {

pv0 {
id = "Vw3IF5-U92i-V2aL-9we1-A4E7-7j1P-4G142u"
device = "/dev/sdc1" # Hint only

status = ["ALLOCATABLE"]
flags = []
dev_size = 199993122 # 95.3642 Gigabytes
pe_start = 384
pe_count = 24413 # 95.3633 Gigabytes
}
}

logical_volumes {

home {
id = "glU5zb-l9I7-SzGg-aXx5-lvgL-on4r-Faqkes"
status = ["READ", "WRITE", "VISIBLE"]
flags = []
segment_count = 1

segment1 {
start_extent = 0
extent_count = 12799 # 49.9961 Gigabytes

type = "striped"
stripe_count = 1 # linear

stripes = [
"pv0", 0
]
}
}
}
}

I went through the backups for my safe volume group, and found the one before I extended the volume group to the (now corrupt) /dev/md3, which is actually the example given above.

Recovering the volume group

It was at this point that I rebooted and held my breath. I brought the system into runlevel 1 (single-user mode) and logged in as root. Out of curiosity I ran some SMART tests on the drives — sda was reporting no partitions, and internal testing eventually reported a persistent read error at about 80% into the drive. It looked like that drive was toast, which would explain why writing to the RAID-5 had failed.

smartctl -t short /dev/sda
sleep 180
smartctl --all /dev/sda

After verifying that it was a hardware issue, I decided to try to point the VG at its previous location (just /dev/sdc1), thereby getting it to forget that the home LV was on the now-inaccessible RAID-5. This could be accomplished by the magic of vgcfgrestore:

# restore the VG metadata for the "safe" VG from the given file
vgcfgrestore -f /etc/lvm/archive/safe_00006.vg safe
# re-scan for VGs, and "safe" should show up as inactive
vgscan
# activate the VG and any LVs inside
vgchange -ay safe
# re-scan for LVs and verify that safe/home exists
lvscan
# not taking any chances -- mount it read-only
mount -o ro /dev/safe/home /home

After the mount succeeded, I began to copy all of the data to my large partition, where it should be safe. I also copied the major bits I didn’t want to lose to a USB flash drive as well, just to be certain. Once that was done, I took a deep breath and rebooted again, this time back to the full system.

Clearing up afterwards

Fortunately, everything worked fine — it was as if nothing had ever gone wrong! All that was left now was to rebuild the broken RAID section.

I decided for safety that I would re-initialise the safe VG’s RAID as RAID-1 rather than RAID-5, so that any two of the drives could fail without losing my precious data:

# stop the RAID array
mdadm -S /dev/md3
# remove the RAID descriptors from the partitions
mdadm --zero-superblock /dev/sda5
mdadm --zero-superblock /dev/sdb5
# create as RAID-1
mdadm -C /dev/md3 -l1 -n3 /dev/sda5 /dev/sdb5 missing

What I hadn’t anticipated was that this would rebuild the RAID-1 using the data from the previous RAID-5… including the LVM descriptors and corrupt content. Once the RAID was online, I ran pvscan and received a warning that the VG metadata was inconsistent. For one heart-stopping moment, it also appeared that LVM thought my home partition was on the corrupt RAID! I quickly stopped the array and thought again.

After some further investigation, I discovered that the first 256 sectors of a partition are used by LVM to hold various information (including multiple copies of the VG metadata). All I had to do to fix this was to destroy that information:

# ensure the array is stopped
mdadm -S /dev/md3
# zero the first 256 sectors
dd if=/dev/zero of=/dev/sda5 bs=512 count=256
dd if=/dev/zero of=/dev/sdb5 bs=512 count=256
# recreate the array -- it may complain that they are already in an array
mdadm -C /dev/md3 -l1 -n3 /dev/sda5 /dev/sdb5 missing

Once this was done, I could run pvcreate /dev/md3 and proceed to extend the VG and pvmove data as before, without worrying that sda would give out on me and cause everything to fail again.

I hope this helps someone!

mdadm -C /dev/md3 -l1 -n3 /dev/sda5 /dev/sdb5 missing

Hearing aid direct audio input

Dave — Mon, 16 Nov 2009 22:59:16 +0000

Many hearing aids can handle direct audio input (DAI), often via a “shoe” that attaches to contacts on the aid itself. This shoe then provides a DAI jack (sometimes called a europlug or eurojack) that cables can be plugged into.

There seems to be a complete lack of cables designed for people with a single hearing aid, so one of my upcoming projects will be to convert a Y-cable that can serve two hearing aids into a joint DAI/earphone cable instead.

DAI connector pinout

The DAI connector has the following pinout:

short/thick pin: ground
long/thin pin: VDD (apparently — doesn’t seem to actually be connected to anything on a 3.5mm cable)
short/thin pin: signal

Remember that a standard 3.5mm audio plug has the following pinout:

tip: left signal
inner sheath: right signal
outer sheath: ground

Next stage: connecting an earphone to the plug on one side of the cable, to see if there are any impedance issues, although the cable I am using (an fmGenie one) is a very low-impedance cable because I have an Oticon hearing aid. Measured with a multimeter, it seems to be around 0Ω, so I think that’s good enough for me. I believe attenuated cables have a 330kΩ resistor in series with each signal line (according to Neil Ferguson’s bluetooth hearing aid hack page).

My ultimate goal is to get hold of a DAI jack that I can solder directly to an old earphone, so that I can just plug the spare DAI cable into the earphone directly, almost as if it were a hearing aid itself. Unfortunately, it seems that it is impossible to get hold of such things. According to Connevans, DAI jacks are only custom-made for hearing aid shoes. After some discussion, it seems like I should be able to make one myself though (with some help!) so we shall have to see how that turns out.

A content-based file manager

Dave — Thu, 04 Jun 2009 22:12:38 +0000

I’ve been thinking recently about Insight again, and I’ve been considering part of the problem with naming and uniqueness.

Names in a traditional file system are made unique based on a full path to the file, but most people think of a file name as just the final component. This would then cause a problem with the move to Insight, as a file could appear in multiple directories, and its only distinguishing feature would be the final component of its path. This is counter-intuitive and can cause all sorts of problems.

Consider makefiles, for example. They rely on a standard named file (Makefile) appearing at various levels in the hierarchy in order to work. Obviously, you would want different makefiles at different levels and in different projects, but Insight as it stands has no way to handle this.

I then started thinking about what makes a file unique. In the end, I came up with two things: name and content. This covers the makefile case (same name, different content) as well as the backup case (same content, different name). It then occurred to me that, in the general case, all you need to distinguish a file is its content, and then actually finding it can all be left up to metadata.

If files are then thought of as containers for data that happen to have a unique internal identifier (which never needs to be exposed to the user, although it can be accessed as, say, the file’s inode number) then the idea of a content-based file manager comes into play. These examples work best with visual media, particularly images, but there is no reason in principle that this could not be extended.

Imagine searching for a file. You know it’s a photo, but you have a large collection of them. With a digital camera and a large-capacity memory card, who needs to ever delete a photo? We’ll assume that you’ve dilligently tagged the photos with metadata as you’ve imported them from the camera, through some easy batch process.

On the tagging point: a lot can be taken from the metadata stored by the camera (date/time, resolution, orientation, black and white/colour, perhaps GPS co-ordinates) and with the right tools, more can be inferred (auto-tagging faces, buildings, perhaps recognising common events like football matches, converting GPS co-ordinates to places, …). As time goes on, people will need to do less and less manual tagging.

Anyway, back to the file manager. You know you are after a picture or a set of pictures. Normal thought processes will probably follow a path similar to: “Yeah, I wanted to show dad those photos from that holiday in Paris that we had two months ago. I think he’d particularly like the ones we got of the Louvre, as well as the ones with me in, of course.” I’ve highlighted various key words that can be translated directly to metadata searches. Notice how these all involve a narrowing down of the query.

To convert these to filters, we then have:

type: photo
holiday
location: Paris
date: two months ago
at least one of:
- location: Louvre
- person: Me

This could also be represented by a query:

type:photo AND holiday AND location:Paris AND date:-2m
AND (location:Louvre OR person:me)

Breaking it down in this way feels fairly technical and wordy, however. I’d much prefer a visual view.

Imagine a black field, speckled with points of light representing your photos:

You filter by “holiday”, and (because it learns based on previous searches) it then groups by location. The ones which have been filtered out fade into nothing, and the photos group into labelled blobs and enlarge slightly:

You filter by date, and as you drag the slider, irrelevant items fade away and relevant ones enlarge:

Then you add the final filters and set the photos up for viewing, perhaps as a slideshow… and you’re done!

Pretty neat, I think.

In-place array uniq in C

Dave — Thu, 10 Jul 2008 21:43:55 +0000

I’ve been developing Insight even though the uni project has come to an end, because it’s fun! I also want to make it more stable and eventually release it under an open-source licence of some kind. There will be an update coming soon, I promise! I now have Internet, so I can write up some things… Anyway, one of the interesting things I wanted to do for Insight was an in-place form of uniq for an array, ideally without any additional memory allocation. It seems that this is something nobody else has done yet! So I set about doing it myself… For those of you who are unfamiliar with the Linux/UNIX command uniq, it takes a sorted list and removes any duplicates. This is almost exactly what I’m trying to do, with one caveat: I need to keep the “discarded” duplicates. What happens is that I have an array containing a number of strings, and these have all been dynamically allocated via malloc() or calloc(). If I just remove or overwrite their pointers, they’ll vanish and cause a memory leak. While I’ve now fixed a large number of leaks thanks to Valgrind, I’m trying to actively avoid any possibility of adding them. Read on for the details…

The Idea

The basic idea is that we have a sorted list of items that may have some duplicates, say:

{A₀, A₁, B₀, C₀, C₁, E₀, F₀, F₁, F₂, G₀, H₀, H₁}

and we want to remove all of the duplicates, so we get two sets:

{A₀, B₀, C₀, E₀, F₀, G₀, H₀} {A₁, C₁, F₁, F₂, H₁}

Note that it doesn’t matter which of the duplicates ends up in which set, just that the items in the first are all unique and that we don’t lose any. Also, A₀ and A₁ have the same value but the subscript will help to distinguish exactly which of the As I’m talking about. Now we have the problem, let’s look at some solutions.

Lossless solution – additional lists

So the first solution uses two temporary lists, unique and duplicate. We start with two pointers into the original list: p and q. We set p to point to the start of the list, and q to point to the element after p. We then copy the item p points at to our unique list. Then, while we still have more items to examine:

If the items pointed to by p and q have the same value, then we copy the item q points at to the duplicate list and advance p and q.
If they are different, we copy the item at q to our unique list, and advance p and q.

We keep doing this until we get to the end of the array, and then copy the unique list to the start of the array, the duplicate list after that (both overwriting the previous contents) and set the number of unique items to the length of the unique

list.

Lossy solution – one list

We can do a similar thing in-place, although this will result in data loss. The essential idea is:

We start in the same way: p pointing to the first element, and q to the second
While q has the same value as p, advance q along the list
When they are different, advance p, copy q to p, and advance q
Once q goes past the end of the list, we’re done, and the offset of p is the number of unique items.

Now this is quite simple, and only requires one pass through the list, but it results in the loss of information, which is unacceptable in this case.

Lossless solution – one list

For this solution, we can also do it with one pass through the list and two pointers. The basic intuition is that we’re going through the array, accumulating a rotating block of stuff that we don’t want. At any given point:

Everything between the start and p (not inclusive) is the unique list (so far)
Everything between p and q (not inclusive) is an unwanted duplicate
Everything between q (inclusive) and the end is yet to be processed.

At the end, we return the number of unique items at the start of the list; everything above that is a duplicate.

For integers

Here is some pseudo-Java that performs my algorithm for a set of integers.

int set_uniq(int set[]) {

  int count = set.length();

  int tmp, p=0, q=1;

  while (p < set.length && q < set.length) {

    while (p < set.length && set[p] != set[q]) {

      p = p + 1;

      if (q < set.length) {

        if (p < q) {

          tmp = set[p];

          set[p] = set[q];

          set[q] = tmp;

        }

        q = q + 1;

      }

      else if (count > p) {

        count = p;

      }

    }

    while (q < max && set[p] == set[q]) {

      q = q + 1;

    }

  }

  // q hit the end and is still the same as p

  if (count > p)

    count = p + 1;



  return count;

}

The general code

The arguments to the general set_uniq() function are very similar to the arguments to standard qsort():

The (sorted) array to work on
The number of items in the array
The size of each item in the array
A comparator function that returns a negative integer if the first argument is less than the second, zero if they are equal, and a positive integer if the first argument is greater than the second.

So, without further ado, my code:

int set_uniq(void *set,

    size_t count,

    size_t elem_size,

    int (*cmp)(const void*, const void*)) {

  void *p, *q, *max;

  void *tmp=calloc(1, elem_size); /* TODO: check for failure */

  p=set;

  q=set+elem_size;

  max=set+(count*elem_size);

  while (p< p-set)

      count = (p-set)/elem_size;

    }

    while (q p-set)

    count = 1 + (p-set)/elem_size;

  ifree(tmp);

  return count;

}

Conclusion

This code is released under a Creative Commons licence (see below). A similar idea will work well for other solutions, like set difference. I’ll post those later if anyone really cares. Also, there is a better way of swapping the items by using the fact that:

a ^= b;

b ^= a;

a ^= b;

swaps a and b without needing a temporary variable. If you just go along the data in 32-bit chunks, performing those XOR operations to swap the data. This actually works out to be 30% or so faster (if I remember correctly). Finally – general code like this is always going to be slower than code that’s been specialised for a particular purpose. Calling this on integers, for example, will be much slower than coding an integer-specific version. I should also point out that you can just overwrite integers, as you don’t need to free them

Insight: An update

Dave — Mon, 09 Jun 2008 16:29:55 +0000

Just a very brief update for the last few days:

Files can now be opened and read – writing and deletion coming soon!
File import does tend to confuse ln, as it expects the new destination to be a symlink rather than a file. Nothing I can really do about this for now though!
I’ve manually assigned some tags to files, and queries now work beautifully (apart from subcategory union, which I hope to tackle later tonight).
Having said that, only simple conjunctive queries work, as there’s no support for negation or disjunction. Yet.
Inode set functions are now in place and working.

One more update before I get back to coding… Insight now has an official logo!

Insight: Where am I now, and where next?

Dave — Thu, 05 Jun 2008 23:18:29 +0000

So I’ve been in Deep Coding Mode™ for quite a while. What have I got to show for it?

Well, the short answer is that Insight is now a functioning file system… for a given definition of “functioning”.

As of this morning:

It can successfully import files from the rest of the hierarchy
Tags (i.e. directories) can be created and removed at any level.
Tags appear and disappear more or less as you expect (i.e. if you already have a tag in your path, it won’t show up in listings again).
Tags that are synonyms will show up as symbolic links to the actual target tag (but currently do not obey the rule in the point above, i.e. if their target has been used in the path, they may still appear).

So far, so good. Now for the limitations:

Files cannot currently be opened, read from, written to, or deleted.
Files must be imported in a strange manner: as absolute symbolic links. They then show up as regular files, although they are actually just links to the originals elsewhere in the filesystem.
Tags cannot be assigned to files (or removed from them)
Files can therefore only be imported at the root level
Queries have no effect on file listing, and so listings just show files in limbo
Of course, there is no subcategory union either.

But I am working on all of these things. At the moment, the main thing is sorting out the internal inode lists. Once those are done, then it should be quite straightforward to do tag assignment/removal and import directly into tags. Plan of action, therefore:

Implement inode insertion/deletion
Implement inode set functions (intersection, union, difference)
Re-implement query tree builder from path. Currently only deals with building a basic conjunctive query tree and assumes that all components are tags. Should:
- Take a path
- Canonicalise it
- Check path components (left-to-right) to ensure tags exist
- If last part is an incomplete tag, treat appropriately
- If last part is a complete tag, then fine
- If last part does not resolve as a tag, then hash it and see if it translates to a known inode
- If not, or if any tags in path do not exist, then path is invalid
- If it is a valid inode, then add QUERY_IS_INODE node to tree
- Otherwise return query tree
Implement query processing:
- Given input set of inodes, produce an output set at each node of the query tree.
- In trivial case with top-level IS_ANY node, output set is the set of limbo inodes, with internal negation flag set to false
- With an IS node, the output set is the recursive union of the inodes belonging to that tag and its subtags, with internal negation flag set to false
- With an IS_NOSUB node, the output set is the set of inodes belonging tag, with internal negation flag set to false
- With an IS_INODE node, the output set contains a single element: the inode.
- With an IS_NOT node with a subquery, the output set is identical to the subquery resultset, with an internal negation flag inverted
- With an IS_NOT node with a tag, the output set is the same as for an IS node, with an internal negation flag set to true
- An AND node output depends on the negation flags of its subqueries:
  - Both false: output is the set intersection of its subqueries, with negation flag clear
  - Both true: output is union of subqueries, with negation flag set
  - Otherwise: output is set difference, with the negation-true set removed from the negation-false set, and the negation flag cleared
- An OR node output depends on the negation flags of its subqueries:
  - Both false: output is union of subquery results, with negation flag clear
  - Both true: output is intersection of subquery results, with negation flag set
  - Otherwise: output is ???
- Probably very likely to be an error if the negation flag is found to be set at the top level.
- Also have to think about how to build a tree from a bracketed expression. But later. Much later.
Output of query processing is an inode set.
Maybe low-overhead query processing just to see if an inode would match the query?
Implement open/read/write as pass-through operations on the inode symlink targets.
Implement symlinking directories as creating synonyms.
Add LOTS of checks.
Note: also have to track inode reference count, so that when it gets to zero the inode is added to the limbo list. Once removed from there, it is removed from the filesystem completely.

These should be quite straightforward to do (I hope), especially as I know more or less exactly what I’m doing. Deadlines are closing in, however, and I have a report and presentation and demo to write yet. Hopefully I can get much of this done by Tuesday, then can spend the day doing bits of my report.

I must say that I do love developing this. It’s just so amazing to be developing a file system and see it work!

Insight: The current plan

Dave — Mon, 19 May 2008 12:51:03 +0000

As it comes time to work on my project again, it’s time to take stock and work out what my plan of action should be.

Looking at the code I have already, I think it’s clear that there is no way I will be able to write a kernel-level file system driver within the 2-3 weeks I have left. Fortunately, I had more or less expected this (as writing kernel code would be likely to take quite a while and be quite complex!) so I’m retreating to my fallback position: a FUSE-wrapped program that will interface with the metadata store.

Also on the cards is the use of Check as a C unit testing framework for my tree code – if I have time. At the moment, getting something to work is far more important than proving it is correct or works in all cases.

Finally, I need to come up with and write the demo programs, and I will shortly be posting about this and then asking the Twitterverse for ideas

Time to enter Deep Coding Mode.

No more exams!

Dave — Fri, 16 May 2008 10:00:09 +0000

So my exams have finally finished… it’s a very surreal feeling. Still got the project to go, but my time at university is even closer to being over. I don’t think it’s really sunk in yet.

Once the exam was over, a group of us headed to get some drinks, and came away with four bottles of cheap champagne sparkling white wine (“We’ve just finished our exams and want quantity over quality. What have you got?”) and headed to the Union to consume it. Sadly things got slightly out of hand there, but I eventually headed back home after completely failing to work out how I could get to Camden (because I managed to completely forget the Tube existed).

Came back after the post-exam celebrations to find these stashed in the kitchen:

My cider has arrived!

So, next stop: project. The due date now seems incredibly close!

Life moves on

Dave — Wed, 23 Apr 2008 00:01:39 +0000

Wow, it’s been a long time since I’ve updated anything here. Insight is moving along, but has currently stalled while I’m in exam mode. I’m not going to talk about that though — this is a rare personal post.

I’ve been looking around for jobs recently, as I will need something to bring in money after I finish at uni in just over two short months. I had originally planned to work for a company based on Slough with one of my housemates, until I received an offer from an ex-Oxford entrepreneur friend. After meeting this friend and his team within just a day or two of the email, I was very excited at the possibility of working with them, especially as I kept having ideas or suggestions for them.

I knew I would have to wait a while for their response, as they had to okay it with their board, and they weren’t planning on deciding on a new hire for some time. As the days fluttered by towards the deadline for this other company and these friends flew out to Silicon Valley for Web Mission, I grew more nervous, hoping that they would get back to me with at least some sort of reply… until yesterday.

I had a phone call (at 7:45am his time) from my main contact there saying they would like to make me an offer! I was overjoyed at this, and was told we’d have a chat later that evening with details. Later that evening, and after some thought, I decided that this definitely was the right path for me, so I resolved to accept the offer.

It is with great pleasure that I announce that I am now the fourth member of the GroupSpaces team I start on or around 30th June, and I’m really, really excited about it! I’ll be moving to Oxford, and living and working there. I can’t wait to start!

Exams begin soon, though. 30th April – 15th May.