Filesystem Storage for MCP Servers

Learn how to implement local filesystem storage for Model Context Protocol servers

Filesystem Storage Integration for MCP

Overview

Local filesystem storage provides a simple and direct way to store model context data for MCP servers. This guide covers implementing a filesystem-based storage provider that follows the MCP storage interface specification.

Prerequisites

Node.js 18 or higher
MCP server base implementation
Read/write permissions for the storage directory

Installation

npm install fs-extra

Implementation

import { promises as fs } from 'fs';
import path from 'path';
import crypto from 'crypto';

class FilesystemStorage implements MCPStorageProvider {
  private baseDir: string;

  constructor(baseDir: string) {
    this.baseDir = baseDir;
  }

  private getContextPath(contextId: string): string {
    // Create hash-based subdirectories to prevent too many files in one directory
    const hash = crypto.createHash('md5').update(contextId).digest('hex');
    const subDir = hash.substring(0, 2);
    return path.join(this.baseDir, subDir, `${contextId}.ctx`);
  }

  async storeContext(contextId: string, data: Buffer): Promise<void> {
    const filePath = this.getContextPath(contextId);
    await fs.mkdir(path.dirname(filePath), { recursive: true });
    await fs.writeFile(filePath, data);
  }

  async retrieveContext(contextId: string): Promise<Buffer> {
    const filePath = this.getContextPath(contextId);
    try {
      return await fs.readFile(filePath);
    } catch (error) {
      if (error.code === 'ENOENT') {
        throw new Error('Context not found');
      }
      throw error;
    }
  }

  async deleteContext(contextId: string): Promise<void> {
    const filePath = this.getContextPath(contextId);
    try {
      await fs.unlink(filePath);
    } catch (error) {
      if (error.code !== 'ENOENT') {
        throw error;
      }
    }
  }
}

Configuration

const storage = new FilesystemStorage('/path/to/storage');

const mcpServer = new MCPServer({
  storage
});

Error Handling

try {
  await storage.storeContext('model-123', contextBuffer);
} catch (error) {
  if (error.code === 'ENOSPC') {
    console.error('Disk space full');
  } else if (error.code === 'EACCES') {
    console.error('Permission denied');
  } else {
    console.error('Storage error:', error);
  }
}

Best Practices

Directory Structure
- Use hashed subdirectories to distribute files
- Implement regular cleanup of temporary files
- Set appropriate file permissions
Performance
- Use asynchronous operations
- Implement caching for frequently accessed contexts
- Monitor disk space usage
Data Integrity
- Implement file checksums
- Use atomic write operations
- Regular backup of storage directory

Monitoring Implementation

class MonitoredFilesystemStorage extends FilesystemStorage {
  async storeContext(contextId: string, data: Buffer): Promise<void> {
    const startTime = Date.now();
    try {
      await super.storeContext(contextId, data);
      metrics.recordStorageOperation('write', Date.now() - startTime);
      metrics.gaugeStorageSize(await this.getCurrentStorageSize());
    } catch (error) {
      metrics.recordStorageError('write');
      throw error;
    }
  }

  private async getCurrentStorageSize(): Promise<number> {
    // Implementation to calculate total storage size
    // ...
  }
}

Testing

describe('FilesystemStorage', () => {
  const testDir = path.join(__dirname, 'test-storage');
  let storage: FilesystemStorage;

  beforeEach(async () => {
    await fs.mkdir(testDir, { recursive: true });
    storage = new FilesystemStorage(testDir);
  });

  afterEach(async () => {
    await fs.rm(testDir, { recursive: true, force: true });
  });

  it('should store and retrieve context', async () => {
    const contextId = 'test-123';
    const testData = Buffer.from('test data');
    
    await storage.storeContext(contextId, testData);
    const retrieved = await storage.retrieveContext(contextId);
    
    expect(retrieved.toString()).toBe(testData.toString());
  });
});

Security Considerations

File Permissions
- Set restrictive file permissions (0600)
- Run process with minimal required privileges
- Sanitize context IDs to prevent path traversal
Data Protection
- Implement encryption at rest
- Secure deletion of sensitive contexts
- Regular security audits

Performance Optimization

Caching

class CachedFilesystemStorage extends FilesystemStorage {
  private cache: Map<string, Buffer>;
  private maxCacheSize: number;

  constructor(baseDir: string, maxCacheSize: number = 100) {
    super(baseDir);
    this.cache = new Map();
    this.maxCacheSize = maxCacheSize;
  }

  async retrieveContext(contextId: string): Promise<Buffer> {
    if (this.cache.has(contextId)) {
      return this.cache.get(contextId)!;
    }
    
    const data = await super.retrieveContext(contextId);
    if (this.cache.size >= this.maxCacheSize) {
      const firstKey = this.cache.keys().next().value;
      this.cache.delete(firstKey);
    }
    this.cache.set(contextId, data);
    return data;
  }
}

Resources

Vector Databases in MCP

Vector databases play a crucial role in the Model Context Protocol (MCP) by enabling efficient storage, retrieval, and querying of high-dimensional vector representations. These representations are often derived from machine learning models and are essential for tasks such as similarity search, recommendation systems, and semantic understanding.