Extending Analysis Types

Overview

The Analysis Agent uses a plugin-based architecture that allows you to add new analysis types without modifying core code. New analysis types are registered in the database and automatically integrated into the system.

This guide walks you through the process of adding a new analysis type, using "Regulatory Analysis" as an example.

Analysis Plugin System

Analysis types are implemented as plugins that conform to a standard interface. Plugins are:

Implemented in code - Plugin code must be written and deployed as part of the application codebase
Registered in the AnalysisTypeRegistry database table (stores metadata, schemas, templates - not code)
Loaded dynamically by the Analysis Agent at runtime (discovers enabled plugins from registry, loads code from codebase)
Configured via the AgentConfig table (type-specific settings)
Executed in parallel with other analysis types
Automatically integrated into content generation (sections generated from plugin templates)

Important: The database registry stores plugin metadata (schemas, templates, enable/disable flags), but the actual plugin implementation code must be part of the deployed codebase. This ensures type safety, security, and proper versioning of plugin logic.

Step-by-Step Guide

Step 1: Define the Analysis Plugin Interface

Your analysis plugin must implement the AnalysisPlugin interface:

interface AnalysisPlugin {
  id: string                    // Unique identifier (e.g., 'regulatory')
  name: string                  // Human-readable name (e.g., 'Regulatory Analysis')
  version: string               // Plugin version (e.g., '1.0.0')
  execute: (
    data: CollectedData,
    config: AnalysisConfig
  ) => Promise<AnalysisResult>
  validateConfig: (config: any) => boolean
  getOutputSchema: () => ZodSchema
  getSectionTemplate?: () => string // Optional: template for content generation
}

Step 2: Create Configuration Schema

Define a Zod schema for your analysis type's configuration:

import { z } from 'zod'

const RegulatoryAnalysisConfigSchema = z.object({
  enabled: z.boolean().default(true),
  sources: z.array(z.enum(['sec', 'fcc', 'fda', 'other'])).default(['sec']),
  minRelevanceScore: z.number().min(0).max(1).default(0.5),
  includeHistorical: z.boolean().default(false),
  ai: z.object({
    model: z.string().default('gpt-4'),
    temperature: z.number().default(0.3),
    maxTokens: z.number().default(2000)
  }).optional()
})

Step 3: Create Output Schema

Define a Zod schema for your analysis type's output:

const RegulatoryAnalysisOutputSchema = z.object({
  type: z.literal('regulatory'),
  results: z.object({
    identifiedRegulations: z.array(z.object({
      id: z.string(),
      title: z.string(),
      agency: z.string(),
      date: z.date(),
      relevanceScore: z.number().min(0).max(1),
      potentialImpact: z.enum(['high', 'medium', 'low']),
      affectedAreas: z.array(z.string()),
      complianceRequirements: z.array(z.string()),
      summary: z.string()
    })),
    complianceStatus: z.enum(['compliant', 'at_risk', 'non_compliant']),
    riskLevel: z.enum(['high', 'medium', 'low']),
    summary: z.string()
  }),
  metadata: z.object({
    executionTime: z.number(),
    confidence: z.number().min(0).max(1),
    dataQuality: z.number().min(0).max(1)
  })
})

Step 4: Implement the Plugin

Create your analysis plugin implementation. Plugin code must be located in the codebase:

Recommended File Structure:

apps/analysis-agent/src/plugins/
  regulatory/
    index.ts          # Plugin implementation (exports AnalysisPlugin)
    schemas.ts        # Configuration and output schemas
    utils.ts          # Helper functions (optional)

Plugin Location Requirements:

Plugins must be in the Analysis Agent codebase (not in database)
Recommended location: plugins/{plugin-id}/index.ts
Plugin must be registered in the plugin loader/registry (see plugins/index.ts)
Plugin ID in code must match the ID registered in AnalysisTypeRegistry table

Create your analysis plugin implementation:

import { CollectedData, AnalysisConfig, AnalysisResult } from '@mediapulse/types'
import { RegulatoryAnalysisConfigSchema, RegulatoryAnalysisOutputSchema } from './schemas'

export const RegulatoryAnalysisPlugin: AnalysisPlugin = {
  id: 'regulatory',
  name: 'Regulatory Analysis',
  version: '1.0.0',
  
  execute: async (data: CollectedData, config: AnalysisConfig) => {
    // Validate configuration
    const validatedConfig = RegulatoryAnalysisConfigSchema.parse(config)
    
    // Your analysis logic here
    const identifiedRegulations = await identifyRegulations(data, validatedConfig)
    const complianceStatus = await assessComplianceStatus(data, identifiedRegulations)
    const riskLevel = calculateRiskLevel(identifiedRegulations, complianceStatus)
    
    // Generate summary using AI
    const summary = await generateSummary(data, identifiedRegulations, validatedConfig)
    
    return {
      type: 'regulatory',
      results: {
        identifiedRegulations,
        complianceStatus,
        riskLevel,
        summary
      },
      metadata: {
        executionTime: Date.now() - startTime,
        confidence: 0.85,
        dataQuality: assessDataQuality(data)
      }
    }
  },
  
  validateConfig: (config: any) => {
    return RegulatoryAnalysisConfigSchema.safeParse(config).success
  },
  
  getOutputSchema: () => RegulatoryAnalysisOutputSchema,
  
  getSectionTemplate: () => {
    return `
      ## Regulatory Analysis
      
      **Compliance Status**: {{complianceStatus}}
      **Risk Level**: {{riskLevel}}
      
      {{summary}}
      
      ### Key Regulations
      {{#each identifiedRegulations}}
      - **{{title}}** ({{agency}})
        - Impact: {{potentialImpact}}
        - Affected Areas: {{affectedAreas}}
        - {{summary}}
      {{/each}}
    `
  }
}

Step 5: Register in Database

INSERT INTO "AnalysisTypeRegistry" (
  id,
  name,
  version,
  description,
  enabled,
  "configSchema",
  "outputSchema",
  "sectionTemplate",
  "createdAt",
  "updatedAt"
) VALUES (
  'regulatory',
  'Regulatory Analysis',
  '1.0.0',
  'Identifies and analyzes regulatory changes that could affect the company',
  true,
  '{"type":"object","properties":{...}}'::jsonb,  -- Your config schema as JSON
  '{"type":"object","properties":{...}}'::jsonb,  -- Your output schema as JSON
  '## Regulatory Analysis...',                    -- Your section template
  NOW(),
  NOW()
);

Or use the admin interface at /admin/agents/analysis-types to register it.

Step 6: Add Configuration

Add configuration for your analysis type in AgentConfig:

UPDATE "AgentConfig"
SET config = jsonb_set(
  config,
  '{analysis,regulatory}',
  '{
    "enabled": true,
    "sources": ["sec", "fcc"],
    "minRelevanceScore": 0.5,
    "includeHistorical": false,
    "ai": {
      "model": "gpt-4",
      "temperature": 0.3,
      "maxTokens": 2000
    }
  }'::jsonb
)
WHERE "agentId" = 'analysis';

Step 7: Map to Content Generation Section

Configure the content generation agent to include a section for your analysis type:

UPDATE "AgentConfig"
SET config = jsonb_set(
  config,
  '{structure,sectionMapping,regulatory}',
  '"regulatorySection"'::jsonb
)
WHERE "agentId" = 'content-generation';

-- Add the section definition
UPDATE "AgentConfig"
SET config = jsonb_set(
  config,
  '{structure,sections}',
  (
    SELECT jsonb_agg(elem)
    FROM jsonb_array_elements(config->'structure'->'sections') elem
    UNION ALL
    SELECT jsonb_build_array(
      jsonb_build_object(
        'id', 'regulatorySection',
        'type', 'analysisSection',
        'analysisType', 'regulatory',
        'enabled', true,
        'maxLength', 500,
        'title', 'Regulatory Updates'
      )
    )
  )::jsonb
)
WHERE "agentId" = 'content-generation';

Step 8: Enable for Tickers/Users

Enable the analysis type for specific tickers or users:

-- Enable for a specific ticker
UPDATE "AgentConfig"
SET config = jsonb_set(
  config,
  '{analysis,enabledTypes}',
  (
    SELECT jsonb_agg(elem)
    FROM jsonb_array_elements(config->'analysis'->'enabledTypes') elem
    UNION ALL
    SELECT jsonb_build_array('regulatory')
  )::jsonb
)
WHERE "agentId" = 'analysis'
AND "tickerId" = 'AAPL';

-- Or enable for all tickers by default
UPDATE "AgentConfig"
SET config = jsonb_set(
  config,
  '{analysis,enabledTypes}',
  (
    SELECT jsonb_agg(elem)
    FROM jsonb_array_elements(config->'analysis'->'enabledTypes') elem
    UNION ALL
    SELECT jsonb_build_array('regulatory')
  )::jsonb
)
WHERE "agentId" = 'analysis'
AND "tickerId" IS NULL;

Step 9: Test the Analysis Type

Manual Testing: Trigger the Analysis Agent manually for a test ticker
Verify Results: Check that results are stored correctly in the database
Verify Content Generation: Ensure the section appears in generated newsletters
Validate Output: Verify output matches your schema

Step 10: Monitor and Iterate

Monitor execution times and success rates
Collect user feedback on the new analysis type
Adjust configuration based on performance
Update the plugin version as you make improvements

Best Practices

1. Naming Conventions

Use lowercase, hyphenated IDs: regulatory-analysis (not RegulatoryAnalysis)
Keep names descriptive but concise
Use semantic versioning for plugin versions

2. Error Handling

Always handle errors gracefully:

execute: async (data: CollectedData, config: AnalysisConfig) => {
  try {
    // Your analysis logic
  } catch (error) {
    // Log error
    // Return partial results if possible
    // Don't fail the entire analysis run
    return {
      type: 'regulatory',
      results: { /* partial results */ },
      metadata: {
        executionTime: 0,
        confidence: 0,
        dataQuality: 0,
        error: error.message
      }
    }
  }
}

3. Performance

Keep execution time under 60 seconds
Use parallel processing where possible
Cache expensive computations
Consider async operations for external API calls

4. Configuration Validation

Always validate configuration:

validateConfig: (config: any) => {
  const result = RegulatoryAnalysisConfigSchema.safeParse(config)
  if (!result.success) {
    console.error('Invalid config:', result.error)
    return false
  }
  return true
}

5. Output Consistency

Always include a summary field
Use consistent metadata fields
Follow the output schema exactly
Include confidence and quality scores

6. Section Templates

Provide clear, readable section templates:

Use markdown for formatting
Include key metrics prominently
Make content scannable for executives
Keep length appropriate (300-500 words)

Removing Analysis Types

To remove an analysis type:

Disable in Registry:

UPDATE "AnalysisTypeRegistry"
SET enabled = false
WHERE id = 'regulatory';

Remove from Configurations:

UPDATE "AgentConfig"
SET config = config #- '{analysis,regulatory}'
WHERE "agentId" = 'analysis';

Remove Section Mapping:

UPDATE "AgentConfig"
SET config = config #- '{structure,sectionMapping,regulatory}'
WHERE "agentId" = 'content-generation';

Remove Section Definition:

UPDATE "AgentConfig"
SET config = jsonb_set(
  config,
  '{structure,sections}',
  (
    SELECT jsonb_agg(elem)
    FROM jsonb_array_elements(config->'structure'->'sections') elem
    WHERE elem->>'analysisType' != 'regulatory'
  )::jsonb
)
WHERE "agentId" = 'content-generation';

The analysis type will no longer be executed, and its sections will not appear in newsletters.

Check AnalysisTypeRegistry.enabled is true
Verify the analysis type is in AgentConfig.analysis.enabledTypes
Check agent logs for plugin loading errors
Verify configuration schema validation passes

Verify section mapping exists in content generation config
Check section is enabled in configuration
Ensure analysis results exist in database
Verify section template is valid

Configuration Errors

Validate configuration against schema
Check for required fields
Verify data types match schema
Review agent logs for validation errors

Next Steps

Review existing analysis types for patterns
Consult the Analysis Agent documentation
Check the Content Generation Agent documentation
Test in experimental environment before production