Test Mode Design for Workflow Engine

Overview

This document describes the design and implementation of a test mode for the workflow engine that maximizes code reuse while providing comprehensive testing capabilities before saving workflows to production.

Design Principles

1. Code Reuse Maximization

Extends the existing ApprovalMLEngine class rather than duplicating logic
Uses the same workflow parsing, condition evaluation, and step execution logic
Maintains identical API interfaces for seamless switching between test and production modes

2. In-Memory Storage

All test workflows, instances, steps, and audit logs are stored in memory
No database writes during test mode execution
Fast execution and easy cleanup

3. Unified Approver Emails

All approvers use the same test email domain (test.flowfrog.app)
Email subjects and bodies clearly indicate workflow steps and context
Easy identification of which step each email corresponds to

4. Comprehensive Audit Trail

Full tracking of step executions, condition evaluations, and approval actions
Detailed logging for debugging and validation
Test summary with execution statistics

Architecture

Core Components

┌─────────────────────────────────────────────────────────────┐
│                    Test Mode Architecture                   │
├─────────────────────────────────────────────────────────────┤
│  ┌─────────────────┐  ┌─────────────────┐  ┌──────────────┐ │
│  │   TestModeContext│  │TestModeWorkflow │  │   API Layer  │ │
│  │   (In-Memory)   │  │     Engine      │  │   (FastAPI)  │ │
│  │                 │  │                 │  │              │ │
│  │ • Instances     │  │ • Extends       │  │ • /test/start│ │
│  │ • Steps         │  │   ApprovalMLEngine│ │ • /test/approve│ │
│  │ • Audit Logs    │  │ • Same Logic    │  │ • /test/status│ │
│  │ • Email Config  │  │ • Test Mode     │  │ • /test/clear│ │
│  └─────────────────┘  │   Overrides     │  └──────────────┘ │
│                       └─────────────────┘                   │
└─────────────────────────────────────────────────────────────┘

Class Hierarchy

ApprovalMLEngine (Base)
    ↓
TestModeWorkflowEngine (Test Mode)
    ├── TestModeContext (In-Memory Storage)
    ├── TestWorkflowInstance
    ├── TestApprovalStep
    └── TestAuditLog

Implementation Details

1. TestModeContext

The TestModeContext class manages all in-memory storage for test mode:

class TestModeContext:
    def __init__(self, test_mode: TestModeEnum = TestModeEnum.SIMULATION):
        self.test_instances: Dict[int, TestWorkflowInstance] = {}
        self.test_steps: Dict[int, TestApprovalStep] = {}
        self.test_audit_logs: Dict[int, TestAuditLog] = {}
        self.test_workflows: Dict[int, Dict[str, Any]] = {}
        
        # Test configuration
        self.test_email_domain = "test.flowfrog.app"
        self.test_approver_email = f"test-approver@{self.test_email_domain}"

Key Features:

Unique ID generation for instances, steps, and audit logs
Test email generation with step identification
Comprehensive tracking of executions and evaluations
Easy data cleanup and reset

2. TestModeWorkflowEngine

Extends the existing ApprovalMLEngine with test mode capabilities:

class TestModeWorkflowEngine(ApprovalMLEngine):
    def __init__(self, db_session: AsyncSession, test_context: Optional[TestModeContext] = None):
        super().__init__(db_session)
        self.test_context = test_context or TestModeContext()
        self.is_test_mode = self.test_context.is_test_mode()

Code Reuse Strategy:

Inherits all core workflow logic from ApprovalMLEngine
Overrides only the database interaction methods
Uses the same condition evaluation and step execution logic
Maintains identical API interfaces

3. Email Generation

Test emails are generated with clear step identification:

def get_test_email_subject(self, workflow_name: str, step_name: str, step_sequence: int) -> str:
    return f"[TEST] {workflow_name} - Step {step_sequence}: {step_name}"

def get_test_email_for_step(self, step_name: str, step_sequence: int, approver_role: Optional[str] = None) -> str:
    role_part = f"-{approver_role}" if approver_role else ""
    return f"test-{step_name}-{step_sequence}{role_part}@{self.test_email_domain}"

Example Email:

To: test-manager-approval-1@test.flowfrog.app
Subject: [TEST] Purchase Approval - Step 1: manager_approval
Body: 
Test Approval Request

Step: manager_approval (Sequence: 1)
Request Data: {"amount": 1500, "department": "IT"}

This is a test workflow execution. Please approve or reject this step.

API Endpoints

Test Workflow Management

Endpoint	Method	Description
`/services/v1/workflows/test/start`	POST	Start a test workflow execution
`/services/v1/workflows/test/{instance_id}`	GET	Get test workflow status
`/services/v1/workflows/test/{instance_id}/approve`	POST	Process test approval action
`/services/v1/workflows/test/summary`	GET	Get test execution summary
`/services/v1/workflows/test/clear`	DELETE	Clear all test data
`/services/v1/workflows/test/instances`	GET	List all test instances
`/services/v1/workflows/test/emails/{instance_id}`	GET	Get test emails for instance

Request/Response Examples

Start Test Workflow:

POST /services/v1/workflows/test/start
{
  "workflow_id": 1,
  "company_id": 999,
  "requestor_user_id": 1,
  "form_data": {
    "amount": 1500,
    "department": "IT",
    "description": "Test purchase request"
  },
  "priority": 1,
  "context_data": {
    "source": "test",
    "environment": "development"
  }
}

Response:

{
  "instance_id": 1,
  "workflow_id": 1,
  "status": "in_progress",
  "steps": [
    {
      "id": 1,
      "step_name": "manager_approval",
      "status": "pending",
      "approver_role": "manager",
      "created_at": "2024-01-15T10:30:00Z"
    }
  ],
  "audit_logs": [...],
  "test_summary": {...}
}

Frontend Integration

WorkflowTestMode Component

The frontend provides a comprehensive test interface:

const WorkflowTestMode = ({ workflow, onClose }) => {
  // Test configuration form
  // Test instances table
  // Current instance details
  // Step approval actions
  // Email preview
  // Test summary
}

Features:

JSON form data input for testing different scenarios
Real-time workflow execution tracking
Step-by-step approval simulation
Email preview with step identification
Comprehensive audit log viewing
Test data management (clear, reset)

Usage Workflow

1. Workflow Design Phase

# Design workflow in YAML
name: Purchase Approval
workflow:
  manager_approval:
    type: decision
    approver: manager
    on_approve:
      continue_to: finance_approval
    on_reject:
      end_workflow: true
  
  finance_approval:
    type: decision
    approver: finance_manager
    on_approve:
      end_workflow: true
    on_reject:
      end_workflow: true

2. Test Mode Execution

# Start test workflow
engine = TestModeWorkflowEngine(db, test_context)
instance = await engine.start_workflow_execution(
    workflow_id=1,
    company_id=999,
    requestor_user_id=1,
    form_data={"amount": 1500, "department": "IT"}
)

# Process approvals
await engine.process_test_approval_action(
    step_id=1,
    action=ApprovalAction.APPROVE,
    user_id=1
)

3. Validation and Debugging

# Get test summary
summary = engine.get_test_summary()
print(f"Executed steps: {summary['executed_steps']}")
print(f"Condition evaluations: {summary['condition_evaluations']}")
print(f"Approval actions: {summary['approval_actions']}")

# View test emails
emails = test_context.get_test_emails_for_instance(instance.id)
for email in emails:
    print(f"To: {email['to']}")
    print(f"Subject: {email['subject']}")
    print(f"Body: {email['body']}")

4. Production Deployment

Once testing is complete and validated:

Workflow is saved to production database
Same engine logic ensures identical behavior
Real emails are sent to actual approvers

Benefits

1. Code Reuse

90%+ code reuse between test and production modes
Same workflow logic ensures consistency
Reduced maintenance overhead
Identical behavior guarantees

2. Comprehensive Testing

Test all workflow paths and conditions
Validate email content and recipients
Debug complex workflow logic
Test edge cases and error scenarios

3. Fast Iteration

No database setup required
Instant workflow execution
Easy data cleanup and reset
Rapid testing cycles

4. Clear Communication

Unified test email domain
Step identification in email subjects
Clear audit trail for debugging
Comprehensive execution summary

5. Production Safety

No risk of affecting production data
Validate workflows before deployment
Test with real workflow logic
Ensure identical behavior

Configuration

Environment Variables

# Test mode configuration
TEST_MODE_ENABLED=true
TEST_EMAIL_DOMAIN=test.flowfrog.app
TEST_COMPANY_ID=999
TEST_USER_ID=1

Test Mode Types

class TestModeEnum(str, Enum):
    DISABLED = "disabled"      # Production mode
    SIMULATION = "simulation"  # Full test simulation
    VALIDATION = "validation"  # Validation only

Future Enhancements

1. Advanced Testing Features

Automated test scenarios
Performance testing
Load testing for complex workflows
Integration testing with external systems

2. Enhanced Debugging

Step-by-step execution visualization
Condition evaluation debugging
Performance profiling
Memory usage tracking

3. Test Data Management

Test data templates
Scenario libraries
Automated test data generation
Test data versioning

4. Integration Testing

Webhook testing
External API integration testing
Email delivery testing
Database integration testing

Conclusion

The test mode design provides a comprehensive solution for testing workflows before production deployment while maximizing code reuse and maintaining consistency between test and production environments. The in-memory storage approach ensures fast execution and easy cleanup, while the unified email system provides clear identification of workflow steps and context.

This design enables rapid iteration and validation of complex workflows, reducing the risk of production issues and ensuring reliable workflow execution.

Test Mode Design for Workflow Engine

Extends the existing ApprovalMLEngine class rather than duplicating logic
Uses the same workflow parsing, condition evaluation, and step execution logic
Maintains identical API interfaces for seamless switching between test and production modes

2. In-Memory Storage

All test workflows, instances, steps, and audit logs are stored in memory
No database writes during test mode execution
Fast execution and easy cleanup

3. Unified Approver Emails

All approvers use the same test email domain (test.flowfrog.app)
Email subjects and bodies clearly indicate workflow steps and context
Easy identification of which step each email corresponds to

4. Comprehensive Audit Trail

Full tracking of step executions, condition evaluations, and approval actions
Detailed logging for debugging and validation
Test summary with execution statistics

Architecture

Core Components

┌─────────────────────────────────────────────────────────────┐
│                    Test Mode Architecture                   │
├─────────────────────────────────────────────────────────────┤
│  ┌─────────────────┐  ┌─────────────────┐  ┌──────────────┐ │
│  │   TestModeContext│  │TestModeWorkflow │  │   API Layer  │ │
│  │   (In-Memory)   │  │     Engine      │  │   (FastAPI)  │ │
│  │                 │  │                 │  │              │ │
│  │ • Instances     │  │ • Extends       │  │ • /test/start│ │
│  │ • Steps         │  │   ApprovalMLEngine│ │ • /test/approve│ │
│  │ • Audit Logs    │  │ • Same Logic    │  │ • /test/status│ │
│  │ • Email Config  │  │ • Test Mode     │  │ • /test/clear│ │
│  └─────────────────┘  │   Overrides     │  └──────────────┘ │
│                       └─────────────────┘                   │
└─────────────────────────────────────────────────────────────┘

Class Hierarchy

ApprovalMLEngine (Base)
    ↓
TestModeWorkflowEngine (Test Mode)
    ├── TestModeContext (In-Memory Storage)
    ├── TestWorkflowInstance
    ├── TestApprovalStep
    └── TestAuditLog

Implementation Details

1. TestModeContext

The TestModeContext class manages all in-memory storage for test mode:

class TestModeContext:
    def __init__(self, test_mode: TestModeEnum = TestModeEnum.SIMULATION):
        self.test_instances: Dict[int, TestWorkflowInstance] = {}
        self.test_steps: Dict[int, TestApprovalStep] = {}
        self.test_audit_logs: Dict[int, TestAuditLog] = {}
        self.test_workflows: Dict[int, Dict[str, Any]] = {}
        
        # Test configuration
        self.test_email_domain = "test.flowfrog.app"
        self.test_approver_email = f"test-approver@{self.test_email_domain}"

Key Features:

Unique ID generation for instances, steps, and audit logs
Test email generation with step identification
Comprehensive tracking of executions and evaluations
Easy data cleanup and reset

2. TestModeWorkflowEngine

Extends the existing ApprovalMLEngine with test mode capabilities:

class TestModeWorkflowEngine(ApprovalMLEngine):
    def __init__(self, db_session: AsyncSession, test_context: Optional[TestModeContext] = None):
        super().__init__(db_session)
        self.test_context = test_context or TestModeContext()
        self.is_test_mode = self.test_context.is_test_mode()

Code Reuse Strategy:

Inherits all core workflow logic from ApprovalMLEngine
Overrides only the database interaction methods
Uses the same condition evaluation and step execution logic
Maintains identical API interfaces

3. Email Generation

Test emails are generated with clear step identification:

def get_test_email_subject(self, workflow_name: str, step_name: str, step_sequence: int) -> str:
    return f"[TEST] {workflow_name} - Step {step_sequence}: {step_name}"

def get_test_email_for_step(self, step_name: str, step_sequence: int, approver_role: Optional[str] = None) -> str:
    role_part = f"-{approver_role}" if approver_role else ""
    return f"test-{step_name}-{step_sequence}{role_part}@{self.test_email_domain}"

Example Email:

To: test-manager-approval-1@test.flowfrog.app
Subject: [TEST] Purchase Approval - Step 1: manager_approval
Body: 
Test Approval Request

Step: manager_approval (Sequence: 1)
Request Data: {"amount": 1500, "department": "IT"}

This is a test workflow execution. Please approve or reject this step.

API Endpoints

Test Workflow Management

Endpoint	Method	Description
`/services/v1/workflows/test/start`	POST	Start a test workflow execution
`/services/v1/workflows/test/{instance_id}`	GET	Get test workflow status
`/services/v1/workflows/test/{instance_id}/approve`	POST	Process test approval action
`/services/v1/workflows/test/summary`	GET	Get test execution summary
`/services/v1/workflows/test/clear`	DELETE	Clear all test data
`/services/v1/workflows/test/instances`	GET	List all test instances
`/services/v1/workflows/test/emails/{instance_id}`	GET	Get test emails for instance

Request/Response Examples

Start Test Workflow:

POST /services/v1/workflows/test/start
{
  "workflow_id": 1,
  "company_id": 999,
  "requestor_user_id": 1,
  "form_data": {
    "amount": 1500,
    "department": "IT",
    "description": "Test purchase request"
  },
  "priority": 1,
  "context_data": {
    "source": "test",
    "environment": "development"
  }
}

Response:

{
  "instance_id": 1,
  "workflow_id": 1,
  "status": "in_progress",
  "steps": [
    {
      "id": 1,
      "step_name": "manager_approval",
      "status": "pending",
      "approver_role": "manager",
      "created_at": "2024-01-15T10:30:00Z"
    }
  ],
  "audit_logs": [...],
  "test_summary": {...}
}

Frontend Integration

WorkflowTestMode Component

The frontend provides a comprehensive test interface:

const WorkflowTestMode = ({ workflow, onClose }) => {
  // Test configuration form
  // Test instances table
  // Current instance details
  // Step approval actions
  // Email preview
  // Test summary
}

Features:

JSON form data input for testing different scenarios
Real-time workflow execution tracking
Step-by-step approval simulation
Email preview with step identification
Comprehensive audit log viewing
Test data management (clear, reset)

Usage Workflow

1. Workflow Design Phase

# Design workflow in YAML
name: Purchase Approval
workflow:
  manager_approval:
    type: decision
    approver: manager
    on_approve:
      continue_to: finance_approval
    on_reject:
      end_workflow: true
  
  finance_approval:
    type: decision
    approver: finance_manager
    on_approve:
      end_workflow: true
    on_reject:
      end_workflow: true

2. Test Mode Execution

# Start test workflow
engine = TestModeWorkflowEngine(db, test_context)
instance = await engine.start_workflow_execution(
    workflow_id=1,
    company_id=999,
    requestor_user_id=1,
    form_data={"amount": 1500, "department": "IT"}
)

# Process approvals
await engine.process_test_approval_action(
    step_id=1,
    action=ApprovalAction.APPROVE,
    user_id=1
)

3. Validation and Debugging

# Get test summary
summary = engine.get_test_summary()
print(f"Executed steps: {summary['executed_steps']}")
print(f"Condition evaluations: {summary['condition_evaluations']}")
print(f"Approval actions: {summary['approval_actions']}")

# View test emails
emails = test_context.get_test_emails_for_instance(instance.id)
for email in emails:
    print(f"To: {email['to']}")
    print(f"Subject: {email['subject']}")
    print(f"Body: {email['body']}")

4. Production Deployment

Once testing is complete and validated:

Workflow is saved to production database
Same engine logic ensures identical behavior
Real emails are sent to actual approvers

Benefits

1. Code Reuse

90%+ code reuse between test and production modes
Same workflow logic ensures consistency
Reduced maintenance overhead
Identical behavior guarantees

2. Comprehensive Testing

Test all workflow paths and conditions
Validate email content and recipients
Debug complex workflow logic
Test edge cases and error scenarios

3. Fast Iteration

No database setup required
Instant workflow execution
Easy data cleanup and reset
Rapid testing cycles

4. Clear Communication

Unified test email domain
Step identification in email subjects
Clear audit trail for debugging
Comprehensive execution summary

5. Production Safety

No risk of affecting production data
Validate workflows before deployment
Test with real workflow logic
Ensure identical behavior

Configuration

Environment Variables

# Test mode configuration
TEST_MODE_ENABLED=true
TEST_EMAIL_DOMAIN=test.flowfrog.app
TEST_COMPANY_ID=999
TEST_USER_ID=1

Test Mode Types

class TestModeEnum(str, Enum):
    DISABLED = "disabled"      # Production mode
    SIMULATION = "simulation"  # Full test simulation
    VALIDATION = "validation"  # Validation only

Future Enhancements

1. Advanced Testing Features

Automated test scenarios
Performance testing
Load testing for complex workflows
Integration testing with external systems

2. Enhanced Debugging

Step-by-step execution visualization
Condition evaluation debugging
Performance profiling
Memory usage tracking

3. Test Data Management

Test data templates
Scenario libraries
Automated test data generation
Test data versioning

4. Integration Testing

Webhook testing
External API integration testing
Email delivery testing
Database integration testing

Conclusion

This design enables rapid iteration and validation of complex workflows, reducing the risk of production issues and ensuring reliable workflow execution.

Test Mode Design for Workflow Engine

On this page

Test Mode Design for Workflow Engine

On this page