Plugin SystemArchitecture

Plugin System Architecture

NetPad’s plugin system provides a comprehensive, scalable architecture for extending workflow capabilities through community-developed and custom plugins. This document outlines the complete architectural design, data flow, and technical implementation.

🏗️ System Overview

The NetPad plugin system is built on a multi-layered architecture that separates concerns between plugin development, distribution, security, and runtime execution:

┌─────────────────────────────────────────────────────────────┐
│                    Plugin Ecosystem                         │
├─────────────────────────────────────────────────────────────┤
│  Developer Tools  │  Plugin Store  │  Runtime Engine       │
│  • CLI SDK        │  • Discovery   │  • Dynamic Registry   │
│  • Templates      │  • Publishing  │  • Shape Integration  │
│  • Testing        │  • Reviews     │  • Execution Context  │
└─────────────────────────────────────────────────────────────┘

📁 Data Storage Architecture

Database Schema

Plugins are stored across multiple MongoDB collections with hierarchical relationships:

1. Plugins Collection

{
  _id: ObjectId,
  name: "unique-plugin-identifier",     // kebab-case
  displayName: "Human Readable Name",
  version: "1.0.0",                     // Semantic versioning
  description: "Plugin description",
  category: "integration|transformation|analysis|utility",
  
  // Ownership & Publishing
  author_id: ObjectId,                  // User who created plugin
  organization_id: ObjectId,            // Optional: org plugin
  scope: "private|organization|public", // Visibility scope
  
  // Plugin Manifest
  manifest: {
    displayName: "Plugin Display Name",
    description: "Detailed description",
    icon: "🔌",                        // Emoji or icon reference
    gradientColors: ["#4CAF50", "#81C784"],
    primaryColor: "#4CAF50",
    
    // Runtime Configuration
    entryPoint: "index.js",
    nodeType: "custom_node_type",
    
    // Permissions Required
    permissions: [
      "network.http",                   // HTTP requests
      "data.read",                      // Read workflow data
      "data.write",                     // Write workflow data
      "context.variables",              // Access context variables
      "memory.read",                    // Read memory
      "memory.write"                    // Write memory
    ],
    
    // Dependencies
    dependencies: {
      "axios": "^1.0.0",
      "lodash": "^4.17.21"
    }
  },
  
  // Plugin Code & Assets
  code: {
    main: "// Plugin implementation code",
    assets: {
      "icon.png": "base64encodeddata",
      "styles.css": "/* CSS styles */"
    }
  },
  
  // Publishing Metadata
  published: true,
  verified: false,                      // Official verification
  featured: false,                      // Featured in store
  
  // Statistics
  stats: {
    downloads: 150,
    rating: 4.5,
    reviews: 23,
    lastDownload: ISODate
  },
  
  // Versioning
  versions: [
    {
      version: "1.0.0",
      published_at: ISODate,
      changelog: "Initial release",
      breaking_changes: false
    }
  ],
  
  // Timestamps
  created_at: ISODate,
  updated_at: ISODate
}

2. Plugin Installations Collection

{
  _id: ObjectId,
  plugin_id: ObjectId,                  // Reference to Plugins collection
  user_id: ObjectId,                    // User who installed
  organization_id: ObjectId,            // Optional: org installation
  
  // Installation Context
  scope: "user|organization",           // Installation scope
  status: "active|disabled|error",      // Current status
  version: "1.0.0",                     // Installed version
  
  // Configuration
  config: {
    apiKey: "encrypted_api_key",        // User-specific config
    enabled: true,
    customSettings: {}
  },
  
  // Usage Tracking
  execution_count: 42,
  last_used: ISODate,
  
  // Update Management
  update_available: false,
  available_version: "1.1.0",
  auto_update: false,
  
  // Timestamps
  installed_at: ISODate,
  updated_at: ISODate
}

3. Plugin Audit Logs Collection

{
  _id: ObjectId,
  plugin_id: ObjectId,
  user_id: ObjectId,
  
  // Audit Information
  action: "install|uninstall|execute|update|configure",
  event_type: "security|performance|error|usage",
  
  // Event Details
  details: {
    version: "1.0.0",
    execution_time_ms: 1250,
    memory_usage_mb: 15.2,
    network_requests: 3,
    errors: [],
    warnings: []
  },
  
  // Security Context
  permissions_used: ["network.http", "data.read"],
  ip_address: "192.168.1.100",
  user_agent: "NetPad/1.0.0",
  
  timestamp: ISODate
}

🔐 Security & Permissions Hierarchy

Permission System

NetPad implements a capability-based security model where plugins must declare required permissions:

// Permission Categories
const PERMISSION_CATEGORIES = {
  NETWORK: {
    "network.http": "Make HTTP requests to external APIs",
    "network.webhook": "Register webhook endpoints",
    "network.dns": "Perform DNS lookups"
  },
  
  DATA: {
    "data.read": "Read workflow input/output data",
    "data.write": "Modify workflow data",
    "data.transform": "Transform data structures"
  },
  
  CONTEXT: {
    "context.variables": "Access workflow context variables",
    "context.secrets": "Access encrypted secrets",
    "context.llm": "Access LLM configuration"
  },
  
  MEMORY: {
    "memory.read": "Read from workflow memory",
    "memory.write": "Write to workflow memory",
    "memory.checkpoint": "Create memory checkpoints"
  },
  
  SYSTEM: {
    "system.file": "File system access (restricted)",
    "system.env": "Environment variable access",
    "system.exec": "Execute system commands (admin only)"
  }
};

Scope Hierarchy

Plugins operate within a hierarchical scope system:

┌─────────────────────────────────────────────────────────────┐
│                    PUBLIC SCOPE                             │
│  • Available to all users                                  │
│  • Published in public store                               │
│  • Requires verification for sensitive permissions         │
│  └─────────────────────────────────────────────────────────┤
│               ORGANIZATION SCOPE                            │
│  • Available to organization members                       │
│  • Internal company plugins                                │
│  • Enhanced permissions for org context                    │
│  └─────────────────────────────────────────────────────────┤
│                 PRIVATE SCOPE                               │
│  • Available only to creator                               │
│  • Development and testing plugins                         │
│  • Full permissions (within user limits)                   │
└─────────────────────────────────────────────────────────────┘

Security Sandboxing

Plugins execute within isolated JavaScript contexts with controlled access:

// Plugin Execution Context
class PluginExecutionContext {
  constructor(plugin, permissions, user) {
    this.plugin = plugin;
    this.allowedPermissions = permissions;
    this.user = user;
    this.sandbox = this.createSandbox();
  }
  
  createSandbox() {
    return {
      // Allowed APIs based on permissions
      fetch: this.allowedPermissions.includes('network.http') ? fetch : null,
      console: new ConstrainedConsole(),
      
      // Plugin-specific context
      netpad: {
        data: new DataAccessor(this.allowedPermissions),
        context: new ContextAccessor(this.allowedPermissions),
        memory: new MemoryAccessor(this.allowedPermissions)
      },
      
      // Blocked APIs
      process: undefined,
      require: undefined,
      global: undefined,
      window: undefined
    };
  }
}

🔄 Runtime Architecture

Dynamic Plugin Registration & Execution

Plugins are registered dynamically at runtime through the PluginRegistryService and executed via the ExecutionEngine:

// Plugin Registration Flow
┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
│   API Request   │───▶│PluginRegistryService│───▶│  Shape Registry │
│ /plugins/install│    │                  │    │                 │
└─────────────────┘    └──────────────────┘    └─────────────────┘
                                │
                                ▼
                       ┌──────────────────┐
                       │   Canvas System  │
                       │ (Auto-recognizes │
                       │  new node types) │
                       └──────────────────┘

Plugin Lifecycle

Node Integration Architecture

Plugins integrate seamlessly with NetPad’s node system:

// Plugin Node Factory
function createPluginNode(plugin) {
  return {
    // Standard Node Properties
    id: generateId(),
    type: plugin.manifest.nodeType,
    x: 0, y: 0,
    width: 220, height: 100,
    
    // Plugin-Specific Properties
    pluginId: plugin.id,
    pluginName: plugin.displayName,
    pluginVersion: plugin.version,
    
    // Visual Styling
    fill: plugin.manifest.primaryColor + '20',
    stroke: plugin.manifest.primaryColor,
    
    // Runtime Configuration
    ports: {
      input: [{ id: 'default', name: 'Input' }],
      output: [{ id: 'result', name: 'Output' }, { id: 'error', name: 'Error' }]
    },
    
    // Plugin Execution Handler
    execute: async (inputs, context) => {
      return await PluginExecutor.run(plugin, inputs, context);
    }
  };
}

Plugin Execution Architecture

Plugins execute through a generic plugin runner that provides standardized interfaces:

// Plugin Execution Flow
┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
│ Workflow Engine │───▶│  Plugin Runner   │───▶│ Plugin Logic    │
│                 │    │ (runPluginNode)  │    │ (by category)   │
└─────────────────┘    └──────────────────┘    └─────────────────┘
         │                       │                        │
         ▼                       ▼                        ▼
┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
│ Input/Output    │    │ Error Handling   │    │ Result Data     │
│ Port Management │    │ & Logging        │    │ & Metadata      │
└─────────────────┘    └──────────────────┘    └─────────────────┘

Plugin Runner Features:

  • Category-Based Execution: Different logic for integration, transformation, testing, and analysis plugins
  • Standardized I/O: Consistent input/output port handling
  • Error Management: Graceful error handling with structured error outputs
  • Performance Monitoring: Execution time tracking and logging
  • Context Integration: Access to workflow context and memory

Execution Categories:

  1. Integration Plugins (category: 'integration')

    • Simulate API calls and external service interactions
    • Return mock responses with configurable endpoints
    • Example: Sample API Plugin

  2. Transformation Plugins (category: 'transformation')

    • Process and transform input data
    • Handle strings, objects, and primitive types
    • Example: Data Transformer Plugin

  3. Testing Plugins (category: 'testing')

    • Validate inputs and system state
    • Perform health checks and debugging
    • Example: Dynamic Test Plugin

  4. Analysis Plugins (category: 'analysis')

    • Analyze data patterns and generate insights
    • Provide recommendations and confidence scores
    • Example: Analysis Node Plugin

Sample Execution Output:

{
  result: {
    data: { /* Plugin-specific results */ },
    status: 'success',
    executionTime: 456
  },
  metadata: {
    pluginId: 'test-plugin-123',
    pluginName: 'Dynamic Test Plugin',
    executionTime: 456,
    timestamp: '2025-01-01T12:00:00.000Z'
  }
}

🚀 Performance & Scaling

Caching Strategy

// Multi-Level Caching
┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   Memory Cache  │    │   Redis Cache   │    │   Database      │
│ (Hot Plugins)   │    │ (Plugin Store)  │    │ (Source of      │
│ 5min TTL        │    │ 1hr TTL         │    │  Truth)         │
└─────────────────┘    └─────────────────┘    └─────────────────┘

Load Balancing

  • Plugin Store: CDN-cached static assets
  • Plugin Registry: Horizontal scaling with consistent hashing
  • Execution Engine: Worker pool isolation for plugin execution

Monitoring

// Plugin Performance Metrics
{
  plugin_id: "sample-api-plugin",
  metrics: {
    execution_time: {
      p50: 150,    // ms
      p95: 500,    // ms
      p99: 1200    // ms
    },
    memory_usage: {
      average: 15.2,  // MB
      peak: 28.7      // MB
    },
    error_rate: 0.02, // 2%
    success_rate: 0.98 // 98%
  }
}

🔧 API Architecture

Plugin Management APIs

GET    /api/plugins                     # Public plugin store
GET    /api/plugins/installed           # User's installed plugins
POST   /api/plugins/{id}/install        # Install plugin
DELETE /api/plugins/{id}/uninstall      # Uninstall plugin
PUT    /api/plugins/{id}/configure      # Configure plugin settings

# Plugin Development APIs
POST   /api/plugins/publish             # Publish new plugin
PUT    /api/plugins/{id}/update         # Update plugin version
GET    /api/plugins/{id}/analytics      # Plugin usage analytics

# Administrative APIs
GET    /api/admin/plugins               # All plugins (admin)
PUT    /api/admin/plugins/{id}/verify   # Verify plugin (admin)
DELETE /api/admin/plugins/{id}          # Remove plugin (admin)

Plugin Execution APIs

POST   /api/mcp/workflow/run            # Execute workflow with plugins
GET    /api/mcp/tools/{pluginId}        # Get plugin tool definition
POST   /api/mcp/command                 # Execute plugin command

🔍 Development Workflow

CLI Development Kit

# Plugin Development Commands
npx create-netpad-plugin my-plugin     # Create new plugin
cd my-plugin
npm run dev                             # Start development server
npm run test                            # Run plugin tests
npm run build                           # Build for production
npm run publish                         # Publish to NetPad store

Plugin Template Structure

my-plugin/
├── package.json                        # Plugin metadata
├── netpad.config.js                    # NetPad configuration
├── src/
│   ├── index.js                        # Main plugin entry
│   ├── node.js                         # Node implementation
│   ├── icon.svg                        # Plugin icon
│   └── styles.css                      # Custom styles
├── tests/
│   ├── unit.test.js                    # Unit tests
│   └── integration.test.js             # Integration tests
└── docs/
    ├── README.md                       # Plugin documentation
    └── examples/                       # Usage examples

🔐 Security Architecture

Threat Model

  1. Malicious Plugins: Sandbox execution, permission controls
  2. Data Exfiltration: Network monitoring, audit logging
  3. Resource Abuse: CPU/memory limits, execution timeouts
  4. Privilege Escalation: Capability-based permissions

Security Controls

// Plugin Security Configuration
const SECURITY_LIMITS = {
  execution: {
    timeout: 30000,           // 30 seconds max
    memory: 50 * 1024 * 1024, // 50MB max
    cpu: 80                   // 80% CPU max
  },
  
  network: {
    maxRequests: 10,          // Per execution
    timeout: 10000,           // 10 seconds
    allowedHosts: [],         // Whitelist (if configured)
    blockedHosts: [           // Blacklist
      'localhost',
      '127.0.0.1',
      '0.0.0.0'
    ]
  },
  
  data: {
    maxPayloadSize: 10 * 1024 * 1024, // 10MB
    allowedMimeTypes: [
      'application/json',
      'text/plain',
      'text/csv'
    ]
  }
};

This architecture provides a secure, scalable, and developer-friendly foundation for NetPad’s plugin ecosystem, enabling unlimited extensibility while maintaining system integrity and user trust.

Migration GuidePublishing & Store