• Home
• Features
• Forums
• Announcements
• Newsletter
• Podcasts
• About
• Write for us

Eclipse File System

This article discusses the Eclipse File System or EFS. Firstly, we'll look at why we have EFS; then, how it's structured. We'll then see an example of using EFS to mount a Zip file (with attached source code for comparison) and finally summarise what EFS provides and a few things to be aware of when creating your own EFS implementation.

In Eclipse 3.2, the way that resources were referenced from disk changed markedly. Ever since its inception as an IDE back in Eclipse 1.0, the way that files are referenced was via a handle called IResource. In fact, IResources, along with the basic subtypes IProject, IFolder and IFile, go to make up the majority of resource-based access inside Eclipse actions, even to this day.

The IResource handle contains meta-data such as the name, time last modified, as well as markers (used to show compile errors and other messages in particular files). Note that IResources are used when caching some elements of the file system; the refresh command in Eclipse synchronises the meta-data contents of the file system with the internal cache of the resource tree. Previously, access was fundamentally file-based; for example, all IResources have the method getLocation(), which was previously expected to be an IPath that points to a full file on disk (and indeed, often used via the .toFile() method to convert it to a java.io.File object). Although this level of abstraction works well for file-based systems, it doesn't translate very well when the data is stored elsewhere (such as on a remote webserver via WebDAV or in an SQL database).

As a result, EFS was created to provide an abstraction of generic file systems by using URIs to represent resources on those file systems, and IResource was updated to include a getLocationURI(). Instead of having to rely on a specific file system (or assume that your operating system will mount it properly), you can supply an EFS implementation and then configure Eclipse to use that for storing data. (There's even a file: EFS implementation, which uses the local file system via java.io.File objects, as well as a null: EFS implementation that does nothing.)

Other code can use EFS programmatically (for example, to copy data up to a remote WebDAV server), and there's no dependency on the resources bundle, so you don't have to use IResource if you just want to interact with other file systems. However, in this article, we're going to look at providing an EFS implementation for the purposes of mounting an archive file, in order to allow us to browse it via the Eclipse navigator.

EFS uses URIs to track files on alternate file systems. The scheme (stuff in front of the colon, like http) is used to select the appropriate EFS implementation, and then the remainder of the URI is parsed by the appropriate file system itself. For example, a WebDAV URI might look like http://www.example.com/dav, whereas an SQL URI might look like sql://db/customer?id=123. Regardless of how the EFS implementation works, every IResource will have a corresponding URI, even if the underlying resource doesn't exist.

An EFS implementation must provide subclasses of FileSystem, which provides information about URI parsing and what the file system is capable of; and FileStore, which represents a single resource/URI on the file system. In addition, the FileStore also exposes attributes about the resource via FileInfo, as well as being able to expose the contents of the file via an InputStream and an OutputStream.

The FileSystem is the factory for translating URIs into FileStores. Of course, being Eclipse, the methods are exposed via interfaces; so the actual contract is spelled out in IFileSystem. There is an abstract FileSystem class that provides most of the key features that can be overridden, which should be used to write new implementations. (Indeed, the Javadoc for IFileSystem says that the interface is not intended to be implemented by clients; instead, the abstract FileSystem should be subclassed instead.)

Each FileSystem has a scheme, which is used to identify URIs that belong to this file system. In the examples above, http and sql would be the schemes.

The capabilities of the FileSystem are exposed by methods attributes(), canWrite() and canDelete(). This allows a file system that cannot be modified (such as a CD-Rom) to expose the fact that no write operations will be permitted. The attributes() method returns a bit-wise or of attribute flags that are provided by this file system; by default, no attributes are known, though you can add whether the system understands read-only files (by returning EFS.ATTRIBUTE_READ_ONLY). The properties view in Eclipse's navigator will not display the read-only checkbox if the file system does not support returning read-only attributes (and similarly for executable or hidden attributes).

Lastly, the FileSystem is used as a factory for obtaining FileStores via individual URIs. A FileStore may perform some caching of previously-accessed URIs, or may return a new FileStore for each one; but obviously some caching may make sense.

The FileStore represents a single file on the remote system. It has a unique URI (which can be parsed in whatever way the file system sees fit) and the file name. In addition, it has ability to acquire an InputStream for reading from the file, as well as an OutputStream which can be used to write to the file (as long as it's not read-only).

The FileStore can have zero to many FileStore children. So, although the FileSystem can be used to translate any URI into a FileStore, the recursive listing is performed by querying the parent FileStore itself. This returns a list of child names, which are then acquired by means of the FileStore to return new FileStore references. It is conventional, but not required, for there to be a relationship between the URIs used. So the children of http://www.example.com/dav might result in http://www.example.com/dav/child, but the children of sql://db/customer?id=123 might result in sql://db/customerDetails?id=456.

The file's meta-data is returned as a separate FileInfo, which encapsulates the file name, the time it was last modified, and any attributes (integer bit-mask, including properties like EFS.ATTRIBUTE_READ_ONLY, EFS.ATTRIBUTE_HIDDEN and EFS.ATTRIBUTE_EXECUTE -- provided that they're a subset of the FileSystem#getAttributes()).

Acquiring a FileInfo will usually necessitate hitting the underlying file system to obtain information. For example, when an editor is activated, it will call fetchInfo() for the purposes of determining whether or not the lastModified time is different from the previous one; if so, the editor asks whether you want the file to be re-loaded. As such, implementing it in an efficient way and using whatever file system caching strategies you can when obtaining this information is pertinent. A WebDAV implementation might cache the ETag, so that when finding out if the file has changed it doesn't need to download the data again, for example.

In order to provide an EFS implementation for Eclipse, we have to provide concrete subclasses of the abstract base classes, and then let Eclipse know by registering the FileSystem as an EFS extension point. For this example, we'll provide a simple implementation of a zip: URI, which will allow us to 'mount' a Zip or Jar archive as a file system in Eclipse (a feature which NetBeans users have had for some time, as I'm sure they'll write in to let me know).

First, we need to create concrete subclasses of the FileSystem and FileStore classes. We don't really need our own FileInfo -- it provides a complete implementation for us to use. We can do this as follows:

• Create a new plug-in project, called "org.example.eclipse.efs.zip"
• Open the manifest, by right-clicking on the project and selecting "PDE Tools -> Open Manifest"
• From the dependencies tab of the manifest, add org.eclipse.core.filesystem and org.eclipse.core.runtime as required bundles
• From the extensions tab of the manifest, add a new extension point org.eclipse.core.filesystem.filesystems
• Add a new child filesystem underneath the extension point, by right-clicking on it and selecting "New -> filesystem" -- the scheme (on the right-hand side) should be zip
• Add a new child run underneath the file system, with a class name of org.example.eclipse.efs.zip.ZipFileSystem
• Save the manifest/plugin.xml
• Create a new class called "org.example.eclipse.efs.zip.ZipFileStore", with a superclass of org.eclipse.core.resources.FileStore
• Create a new class called "org.example.eclipse.efs.zip.ZipFileSystem", with a superclass of org.eclipse.core.resources.FileSystem

You should end up with a MANIFEST.MF and plugin.xml that looks something like the following:

Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: ZipEFS Plug-in
Bundle-SymbolicName: org.example.eclipse.efs.zip;singleton:=true
Bundle-Version: 1.0.0
Bundle-Vendor: Alex Blewitt
Require-Bundle: org.eclipse.core.filesystem,
 org.eclipse.core.runtime

<?xml version="1.0" encoding="UTF-8"?><?eclipse version="3.2"?>
<plugin>
   <extension point="org.eclipse.core.filesystem.filesystems">
      <filesystem scheme="zip">
         <run class="org.example.eclipse.efs.zip.ZipFileSystem"/>
      </filesystem>
   </extension>
</plugin>

There's good support in Java for acquiring Zip and Jar files, primarily because all Java archives are based on the Zip standard. Although there are separate classes in the class libraries for manipulating Zip and Jar files, they're pretty much interchangeable, at least for the purpose of this exercise. So we'll delegate the work to the class libraries. You might want to read through the Javadoc for the ZipFile and ZipEntry classes if you've not come across them before.

Unfortunately, the Zip file format provides entries in a list-based format instead of a tree-based format. In order to convert entries of the form "org/example/Test.class" into a tree-based representation "org { example { Test.class, Test2.class } }" there's an implementation in the sample code of a ZipItem superclass, with corresponding ZipRootItem, ZipDirectoryItem and ZipFileItem subclasses. In this example, the org and example would be represented as ZipDirectoryItems, whilst the Test.class and Test2.class would be ZipFileItems. The implementation of these utility classes is outside the scope of this article, but you can investigate what it's doing in the supplied source code.

Since all resources in an EFS provider are represented by URI, we'll use URIs of the format zip:/path/to/file/in?file:///the/file.zip. This has an immediate advantage that we can use the URI's methods getPath() and getQuery() to return the entry and the location of the actual Zip file, respectively.

We've already told EFS that the scheme is zip, via the extension point. Whenever it hits a zip URI, it will delegate to the ZipFileSystem class. The only method that must be implemented is the getFileStore(URI) method, which will have been populated for you if you had the 'Add required methods' checkbox selected when adding the classes in the first place. If not, you can add them by choosing the quick fix bulb or Ctrl+1 on the error, or by using Source -> Override/Implement methods (which automatically selects abstract methods by default).

Although there are other methods which can be overridden for performance, we'll just delegate the work of the factory method to a new ZipFileStore() constructor:

public IFileStore getStore(URI uri) {
  String name = "root";
  ZipItem parent = null;
  return new ZipFileStore(name,uri,parent);
}

In a real EFS implementation, you'd want to do something a bit more resource-efficient than creating a new ZipFileStore all the time; caching is left as an exercise for the reader, or from supplied code.

The ZipFileStore represents an element in the resource hierarchy. Each FileStore is parented by another FileStore; although there's nothing preventing them being mixed, we'll assume that a ZipFileStore is parented by another ZipFileStore, and that the root will have a parent of null. As a result, we pass in the local name and parent when building a ZipFileStore, since we'll need those later:

private String name;
private ZipFileStore parent;

private ZipFileStore(String name, ZipFileStore parent) {
  this.name = name;
  this.parent = parent;
}
public IFileStore getParent() {
  return parent;
}
public String getName() {
  return name;
}

Of course, we need to ensure that we can populate our ZipFileStore with some information; so we'll need to obtain a zip file, along with its contents somehow. We can associate it with each ZipFileStore:

private ZipItem item;

public ZipFileStore(String name, ZipFileStore parent, ZipItem item) {
  this(name,parent);
  this.item = item;
}

We need to arrange for the ZipItem to be acquired when the root of a new ZipFileStore tree is made. We can do this by decoding the URI to get the query, and then using that to instantiate a new File, and subsequently a new ZipRootItem:

URI uri = ...                                       // from ZipFileSystem.getURI()
URI zipFileURI = uri.getQuery();                    // pulls off (zip:....?)file://path/to/the.zip
IFileStore zipFileStore = EFS.getStore(zipFileURI); // we can use EFS to find the file:// URI :-)
File zipFile = zipFileStore.toLocalFile(0,null);    // convert it to a File
ZipItem item = new ZipRootItem(zipFile);            // and then make a new root from it

Now that we've got our root, we need to tell EFS how to find information about the node itself. We can simplify coding a little if we have a helper method to determine whether the item is a directory or not; we'll also need to return information about the node via a FileInfo object. Amongst other things, this contains the last modified time, whether it's read-only, hidden, or executable, and so on:

boolean isDirectory() {
  return item instanceof ZipDirectoryItem;
}

public IFileInfo fetchInfo(int options, IProgressMonitor monitor) {
  FileInfo info = new FileInfo(getName());
  if (isDirectory()) {
    info.setDirectory(true);
  } else {
    info.setDirectory(false);
    info.setLastModified(((ZipFileItem) item).getLastModified());
  }
  info.setExists(true);
  info.setAttribute(EFS.ATTRIBUTE_READ_ONLY, true);
  return info;
}

Now that we're reporting back some information about the current node, it's time to determine what the children are. These will only get called for directories, and the childNames() will return a list of Strings that are considered child names for this node:

public String[] childNames(int options, IProgressMonitor monitor)
  throws CoreException {
  if (isDirectory()) {
    Collection collection = ((ZipDirectoryItem) item).getChildren();
    String[] children = new String[collection.size()];
    Iterator iterator = collection.iterator();
    int i = 0;
    while (iterator.hasNext()) {
      ZipItem child = (ZipItem) iterator.next();
      children[i++] = child.getName();
    }
    return children;
  } else {
    return new String[0];
  }
}

To obtain the FileStore for each child, there's a getChild() method which takes a String name and returns a new node. As well as hooking up the item, we'll also maintain the parent-child relationship (as before, caching is left for the reader):

public IFileStore getChild(String name) {
  if (isDirectory()) {
      ZipItem child = ((ZipDirectoryItem) item).getItem(name);
      return new ZipFileStore(name, this, child);
  } else {
    return null;
  }
}

Of course, all this boils down to being able to get the contents of a file, which is done by the openInputStream() method (which delegates to ZipItem):

public InputStream openInputStream(int options, IProgressMonitor monitor)
  throws CoreException {
  if (!isDirectory()) {
    try {
      return ((ZipFileItem) item).openInputStream();
    } catch (IOException e) {
      throw new CoreException(new Status(IStatus.ERROR,"org.example.efs.zip","Failed to open file",e));
    }
  } else {
    return null;
  }
}

To test the plugin, like any other Eclipse plugin, we need to launch a runtime instance of Eclipse. It's easiest to use the workbench; by going to the launch configuration from the Run... -> Run menu, you can add a new Eclipse Application. Running the product org.eclipse.sdk.ide or the application org.eclipse.ui.ide.workbench, with all plugins enabled, will be enough to test this out.

There's no UI for creating a linked resource at the moment, so if you want to create your own link you'll have to supply your own action or code to do it. Fortunately, all links are stored in the .project file, and it's easy to add our own manually. Create a new project called TestEFS, and once created, switch to the Navigator perspective (specifically, the Navigator view). You should see a .project file that looks something like the one below. In order to hook up your file system to a file called /path/to/a.zip, simply cut'n'paste the code in bold into your .project file:

<?xml version="1.0" encoding="UTF-8"?>
<projectDescription>
  <name>Test</name>
  <comment></comment>
  <projects>
  </projects>
  <buildSpec>
  </buildSpec>
  <natures>
  </natures>
  <linkedResources> 
    <link> 
    <name>AZip</name> 
    <type>2</type> 
    <locationURI>zip:/?file:////path/to/a.zip</locationURI> 
    </link> 
  </linkedResources>
</projectDescription>

Of course, there are programmatic ways of creating linked resources such as using IFile or IFolder's createLink() method, which would be the correct way to do it in code. The example here merely illustrates how the linked resource is stored, and to show the resource's URI easily.

What you should see, after doing this, is a folder underneath your TestEFS project called AZip, and underneath that the contents of your zip file located at /path/to/a.zip. Note that if you're running on Windows, you still need to put the file URI with the / slash; and if it's on a different drive, you may need to use file:///c:/path/to/a.zip.

Although this is a read-only file system, it should be possible for you to drill down into the zip file and open the editors for the file contents. If it were a read-write system, you'd be able to write data back by implementing a openOutputStream() on the ZipFileStore, but that's outside the scope of this article.

The EFS is an abstraction of file systems, and allows you to relatively easily build your own implementation of file systems. Not only does it allow the navigator to use different systems, any other program could call EFS to acquire files from remote systems (like the local file:/// was used by this example). It can also be used by other Eclipse-platform or OSGi applications since it has a minimal set of dependencies (although it does depend on Java 1.4 due to reliance on the URI class).

There's a few gotchas that can catch you unawares with EFS. Firstly, because resources now map to URIs, it's not necessarily the case that every IResource has a corresponding local file, so code that uses IResource.getLocation() will have to defend against a null return result. Prior to Eclipse 3.2, this would always result in a Path object (and often converted to a File when developers were trying to acquire resources from a bundle). Now, the IResource.getLocationURI() should be used, and EFS interrogated to find out the data.

Some of the messages are a bit bizzare. For example, when a file is being copied from one directory to another on an EFS-backed location, the data is copied (with the internal transferStreams() method called via the copyFile() method), and then the attributes are copied (with the internal transferAttributes() method called from the same copyFile() method). However, transferAttributes() ends up calling putInfo(), and the default implementation of putInfo() is to throw an exception saying that the file system is read-only, even when it's not.

Not all of the Eclipse 3.2 API was migrated over to using the new getLocationURI(); a few bugs remain in 3.2.1 and may not be back-ported from their fixes in 3.3 to 3.2.2. For example, copying files between EFS had a minor bug, as did local change history on EFS files. Although these are relatively minor, they're something to be aware of if you need to roll out EFS functionality across a 3.2 client base.

Efficient caching of file stores is a difficult thing to do right. (The example code provided certainly doesn't do a good job of caching and noticing when things have changed.) For light-weight file systems, it may not be necessary to do caching; but if your connections are to a remote HTTP server or SQL server, you may need to more aggressively cache the results. It's not unknown for the fetchInfo() to be invoked multiple times during some operations; one optimisation is to store the timestamp of the last calculated FileInfo, and then continue to return that until timestamp+delta has passed.

The API does provide more optimisations for you to take advantage of if necessary. For example, there's a method childInfos() to obtain all the FileInfos in a single hit, as well as childStores(). Similarly, copies or moves may be more efficiently implemented in bulk operations.

Although there are several other virtual file systems for Java already (Apache VFS being the most obvious) the goal to fit in with the IResource in a light-weight fashion and not add too much was a key goal. Additionally, the file system needed to allow extra file systems to be added by contribution to Eclipse's extension registry (such as the one developed in this article), which existing solutions wouldn't have been able to do directly. The discussion about the pros and cons is listed at bug 106176 if you'd like to read the commentary behind some of these other approaches.

It's also worth noting that there are some example EFS implementations available at Eclipse.org, which are linked from the EFS wiki page, including a different (i.e. better) implementation of a Zip file system. In addition, the Eclipse example has an action which allows the Zip file to be mounted and unmounted from within the navigator view, which is a lot more user-friendly than editing the project file. (The main reason for showing the .project editing is to show how linked files are represented in Eclipse.)

If you'd like to find out more, you can read up on the EFS Javadoc, or try out the attached sample code. Any questions, comments or replies can be posted to the corresponding forum on EclipseZone.

Thanks are due to John Arthorne and Neil Bartlett who provided useful feedback about this article prior to publishing.

We encourage you to ask the author any questions or discuss the article here.